AN  INVESTIGATION  OF  THE  EFFECTS  OF  BOUNDARY  AVOIDANCE  ON 

PILOT  TRACKING 


THESIS 


Randy  D.  Warren,  Captain,  USAF 

AFIT/GAE/ENY/06-S 1 1 

DEPARTMENT  OF  THE  AIR  FORCE 
AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 


Wright-Patterson  Air  Force  Base,  Ohio 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 


The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official 
policy  or  position  of  the  United  States  Air  Force,  Department  of  Defense,  or  the  U.S. 


Government. 


AFIT/GAE/ENY/06-S1 1 


AN  INVESTIGATION  OF  THE  EFFECTS  OF  BOUNDARY  AVOIDANCE  ON 

PILOT  TRACKING 

THESIS 

Presented  to  the  Faculty 
Department  of  Aeronautics  and  Astronautics 
Graduate  School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 
Air  University 

Air  Education  and  Training  Command 
In  Partial  Fulfillment  of  the  Requirements  for  the 
Degree  of  Master  of  Science  in  Aeronautical  Engineering 

Randy  D.  Warren,  BS 
Captain,  USAF 

September  2006 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 


AFIT/GAE/ENY/06-S1 1 


AN  INVESTIGATION  OF  THE  EFFECTS  OF  BOUNDARY  AVOIDANCE  ON 

PILOT  TRACKING 


Randy  D.  Warren,  BS 
Captain,  USAF 


Approved: 


_ /signed/ _  _ 

MAJ  PAUL  A.  BLUE  (Thesis  Advisor)  Date 

_ /signed/ _  _ 

DR.  BRADLEY  LIEBST  (Committee  Member)  Date 

_ /signed/ _  _ 

DR.  DONALD  L.  KUNZ  (Committee  Member)  Date 

_ /signed/ _  _ 

LT  COL  DANIEL  R.  MILLMAN,  Ph.D.  (Committee  Member)  Date 


AFIT/GAE/ENY/06-S1 1 


Abstract 

Pilot-Involved  Oscillations  (PIOs)  remain  a  significant  issue  in  the  design,  testing 
and  operations  of  aerospace  vehicles.  Traditional  methods  for  predicting,  describing,  and 
analyzing  these  events  have  provided  the  community  with  improved  methods  for 
minimizing  the  occurrences  of  PIOs.  However,  these  events  continue  to  occur  over  a 
wide  range  of  aerospace  vehicle  types  and  over  a  wide  range  of  pilot  acumen. 

The  introduction  of  boundary  avoidance  tracking  (BAT)  by  Mr.  William  Gray  in 
2004  added  a  missing  piece  to  the  PIO  puzzle.  This  theory  presented  that  PIOs  may 
result  from  increasing  pilot  gain  resulting  not  from  maintaining  a  specified  condition,  but 
avoiding  imposed  limits  or  boundaries  on  a  specified  task.  The  initial  modeling  and 
simulation  conducted  by  Mr.  Gray  has  provided  the  community  with  a  starting  point  for 
applying  this  theory  to  the  analysis  of  PIO  events. 

This  thesis  characterizes  BAT  in  the  dynamic  flight  environment.  Through  the 
analysis  of  repeated  BAT  events  in  a  T-38C  aircraft,  initial  characteristic  parameters  for 
BAT  have  been  identified  and  developed.  The  key  BAT  parameters  were  found  to  be 
independent  of  pilot  and  exhibited  some  dependence  on  aircraft  load  factor.  Overall, 
BAT  was  successfully  demonstrated  and  characterized  during  this  research  and  the 
results  will  provide  the  community  with  a  better  understanding  of  the  role  BAT  plays  in 
PIO  prediction  and  analysis. 
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AN  INVESTIGATION  OF  THE  EFFECTS  OF  BOUNDARY  AVOIDANCE  ON 

PILOT  TRACKING 


1.  Introduction 


1.1  Motivation 

Pilot-Involved  Oscillations  (PIOs)  continue  to  present  significant  problems  to  the 
aviation  community.  Since  the  advent  of  powered  flight,  pilots  have  reported  undesirable 
aircraft  oscillatory  behavior.  In  fact,  as  aircraft  have  grown  in  complexity,  so  too  have 
the  variety,  complexity  and  frequency  of  PIOs  (McRuer,  1995:3).  The  continued  need  to 
develop  the  understanding  and  characterization  of  this  handling  qualities  problem  has 
been  made  very  apparent  with  several  high-profile  PIO  events. 

The  YF-22  suffered  a  destructive  PIO  in  1992  during  an  approach  to  a  go-around; 
two  JAS-39  Gripen  aircraft  were  lost  to  PIOs  during  approaches  in  1990  and  1993 
(McRuer,  1995:9);  and  the  C-17A  experienced  several  PIOs  during  Dutch  roll  testing, 
aerial-refueling  tracking,  and  three-engine  approaches  and  landings  (Preston  and  others, 
1996:20-49). 

Currently,  modeling  and  testing  for  PIO  requires  a  pilot  to  perform  some  sort  of 
specified  task  in  order  to  close  the  pilot  plus  aircraft  loop  and  evaluate  the  system 
handling  qualities.  For  example,  the  tasks  could  include  aerial  refueling,  formation  flight, 
precision  approaches  to  spot  landings,  or  many  others  (McRuer,  1995:4).  During  both 
modeling  and  testing,  the  pilot  must  minimize  the  error  associated  with  the  task; 
however,  this  error  is  not  the  only  thing  the  pilot  is  tracking.  Every  task  will  also  have 
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associated  boundaries  or  limits.  Whether  these  boundaries  are  placed  on  the  task  itself  or 
are  a  function  of  the  pilot  or  aircraft,  they  are  a  part  of  the  scenario.  Until  now,  these 
boundaries  have  largely  been  ignored  in  the  prediction  and  analysis  of  task  performance 
and  PIO  susceptibility. 

1.2  Tracking  Tasks 

As  previously  stated,  handling  qualities  are  evaluated  utilizing  some  specified  task. 
That  task  has  a  desired  condition  that  must  be  maintained  and  also  some  set  of 
boundaries.  Figure  1  illustrates  an  arbitrary  tracking  task.  The  figure  depicts  an 
illustration  of  an  aircraft  Heads-Up  Display  (HUD)  set  in  a  ground  attack  mode.  The 
center  circle  with  a  dot  in  the  middle  represents  the  aircraft’s  flight  path  vector.  The 
circle  at  the  bottom  connected  to  a  line  represents  the  impact  point  if  a  bomb  was  released 
at  that  moment. 

1.2.1  Point  Tracking 

The  specified  task  for  this  illustration  is  for  the  pilot  to  hit  the  designated  target 
area.  To  do  so,  the  pilot  must  maneuver  the  aircraft  such  that  the  bottom  circle  lines  up 
over  the  designated  target  area.  This  is  called  the  point  tracking  task. 
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Figure  1.  Tracking  Task  Illustration 


1.2.2  Boundary  Avoidance  Tracking 

The  boundaries  imposed  on  this  task  are  the  buildings  located  on  each  side  of  the 
designated  target  area.  The  pilot  must  not  release  his  weapon  onto  either  of  these 
structures.  As  the  pilot  begins  to  approach  one  of  these  boundaries,  the  aircraft  will  be 
controlled  away  from  the  boundary  to  avoid  hitting  that  boundary.  This  is  called 
boundary  avoidance  tracking  (BAT).  The  pilot  is  still  performing  the  point  tracking  task, 
but  inputs  to  the  system  are  made  to  avoid  a  perceived  approach  to  a  boundary,  not  to 
minimize  the  error  associated  with  the  designated  target  area. 
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For  BAT,  two  parameters  are  used  to  define  the  pilot’s  awareness  of  the 
boundary.  First  is  distance  to  boundary,  D boundary ■  This  is  the  distance  the  pilot  perceives 
before  a  boundary  is  crossed  (Figure  1).  Second  is  the  time  to  boundary,  boundary .  This 
time  is  determined  from  the  distance  to  boundary  and  the  time  rate  of  change  of  the 
distance  to  boundary,  rboundary ■  It  is  believed  that  the  primary  mechanism  for  boundary 
awareness  and  avoidance  is  the  pilot’s  perceived  time  to  boundary.  This  will  be 
discussed  further  in  Chapter  2. 

1,2.3  Comparison  of  Point  Tracking  and  Boundary  Avoidance  Tracking 

Though  these  two  types  of  tracking  seem  somewhat  similar,  they  are  in  essence 
the  reverse  of  each  other.  With  point  tracking,  pilots  tend  to  decrease  their  level  of  inputs 
as  they  approach  the  specified  condition  and  increase  as  they  depart  the  condition.  In 
BAT,  they  increase  their  level  of  inputs  to  avoid  a  boundary  and  decrease  their  inputs 
when  departing  the  boundary. 

As  such,  it  is  critical  to  understanding  the  boundaries  levied  on  any  given  task. 
Since  all  tasks  are  bounded  by  mission,  operational,  structural,  and  aerodynamic  limits,  it 
is  critical  to  understand  how  close  to  these  boundaries  a  pilot  may  have  to  operate  for  the 
specified  task. 

1.3  Research  Objectives 

The  focus  of  this  research  was  to  investigate  the  effect  that  BAT  has  on  the  ability 
of  a  pilot  to  accomplish  a  specified  task.  To  determine  this  effect,  BAT  was 
characterized  in  a  dynamic  flight  environment.  Once  it  was  adequately  characterized,  the 
relationship  between  BAT  and  aircraft  flying  qualities  could  be  investigated. 
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Development  of  this  relationship  will  lead  to  a  better  understanding  of  how  BAT  and 
aircraft  flying  qualities  work  together  to  affect  the  ability  of  the  pilot  to  successfully 
accomplish  a  specified  task. 

1.4  Experiment  Overview 

BAT  data  were  collected  in  the  flight  environment  by  flying  two  T-38  aircraft  in 
formation  with  the  test  pilot  maintaining  a  specified  condition.  The  task  was  then 
bounded  and  the  test  pilot  was  to  continue  to  maintain  the  specified  condition  while 
avoiding  the  boundaries.  Data  were  collected  using  both  an  aircraft  data  acquisition 
system  and  a  video  camera  system.  Aircraft  configuration  and  flight  conditions  were 
varied  to  present  as  wide  a  range  as  possible  of  aircraft  flying  qualities  and  multiple 
techniques  to  elicit  BAT  were  developed  and  employed. 

1.5  Preview  of  Results 

While  no  definitive  correlation  between  aircraft  flying  qualities  and  the  BAT 
parameters  was  found,  the  data  obtained  led  to  further  characterization  of  human 
perception  and  reaction  to  boundaries.  Two  parameters  were  identified  as  apparent 
ingrained  human  characteristics  -  the  time  to  boundary  for  maximum  BAT  feedback  gain 
and  the  time  elapsed  between  minimum  and  maximum  feedback  gain.  The  time  to 
boundary  for  minimum  BAT  feedback  gain  was  found  to  be  a  function  of  aircraft  load 
factor.  Frequency  domain  analysis  also  revealed  that  pilots  operated  with  very  little 
phase  margin  during  BAT  events  leading  to  pilot-involved  oscillations  (PIOs). 
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1.6  Thesis  Overview 


In  Chapter  2,  a  detailed  review  of  the  previous  work  conducted  by  Mr.  William 
Gray  is  provided.  The  foundation  for  the  BAT  theory  developed  through  his  modeling 
and  simulation  work  are  key  to  understanding  the  experiment  methodology  and  results  of 
this  thesis.  Chapter  3  details  the  experiment  methodology  and  instrumentation  utilized  to 
obtain  the  data  during  the  research.  Chapter  4  presents  the  data  reduction  and  analysis 
and  Chapter  5  summarizes  the  results,  lessons  learned,  and  recommendations. 
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2.  Background 


2.1  Boundary  Avoidance  Tracking  Introduction 

The  concept  of  BAT  was  first  introduced  by  Mr.  William  Gray  of  the  United 
States  Air  Force  Test  Pilot  School  to  the  Society  of  Experimental  Test  Pilots  in  2004 
(Gray,  2004:3).  In  his  paper,  Mr.  Gray  proposed  a  mechanism  for  PIOs  that  was 
markedly  different  from  the  traditional  PIO  assumptions  that  have  governed  the 
prediction  and  prevention  of  PIOs  over  the  past  several  decades  (McRuer,  1995:14-34). 
That  is,  some  PIOs  may  be  explained  by  the  pilot  successively  controlling  the  aircraft  to 
avoid  opposing  boundaries  as  opposed  to  over-controlling  while  point  tracking. 

The  United  States  Department  of  Defense  (DoD)  defines  PIO  as  “sustained  or 
uncontrollable  oscillations  resulting  from  the  efforts  of  the  pilot  to  control  the  aircraft” 
(MIL-HDBK-1797,  1997: 15 1).  Of  note  is  that  Mr.  Gray’s  theory  does  not  contradict  this 
definition.  In  fact,  the  resulting  oscillatory  motion  from  repeated  BAT  events  meets  this 
definition  of  a  PIO.  This  means  that  past  PIOs  may  have  exhibited  BAT  characteristics 
but  have  been  unsuccessfully  analyzed  because  point  tracking  techniques  have  been 
applied. 

2.1.1  Point  Tracking 

In  traditional  pilot-aircraft  system  analysis,  a  point  tracking  task  is  specified  and 
the  pilot  is  required  to  maintain  that  condition.  The  pilot  will  provide  system  inputs  when 
a  tracking  error  is  perceived  in  an  effort  to  return  the  system  to  the  specified  condition. 
The  pilot’s  experience  level,  the  difficulty  of  the  task,  and  the  aircraft’s  handling  qualities 


7 


(among  many  other  factors)  are  all  responsible  for  the  pilot’s  overall  perfonnance  in 
maintaining  the  specified  condition. 

Undesirable  aircraft  oscillatory  motion  is  generally  excited  by  the  level  of  pilot 
input  into  the  aircraft  system.  Through  increasingly  stringent  levels  of  tracking  error,  the 
pilot  is  driven  to  a  state  of  zero  tracking  error  in  an  effort  to  maintain  the  specified 
condition.  As  this  state  is  approached,  pilot  inputs  increase  in  frequency  and  in  some 
cases  in  amplitude  as  well.  Once  the  zero  error  state  is  reached  and  the  pilot  is  tracking 
aggressively  and  assiduously,  the  system  experiences  the  highest  frequency  and 
amplitude  inputs.  It  is  at  this  state  that  any  undesirable  aircraft  tendencies  are  generally 
identified  (Brown,  2002:23-1). 

2.1.2  Boundary  Avoidance  Tracking 

For  BAT,  the  pilot  is  no  longer  providing  system  inputs  to  maintain  a  specified 
condition,  but  is  instead  providing  inputs  in  an  attempt  to  avoid  an  approaching 
boundary.  At  some  point,  the  pilot  becomes  aware  of  the  boundary  and  begins  making 
system  inputs  in  an  effort  to  stop  encroaching  on  the  boundary.  At  this  point,  the  pilot 
has  abandoned  tracking  the  specified  condition  and  is  now  attempting  to  avoid  the 
approaching  boundary. 

If  the  pilot  perceives  that  the  boundary  is  successfully  avoided,  then  he  returns  to 
tracking  the  specified  condition.  If  the  pilot  perceives  a  continued  approach  to  the 
boundary,  then  the  level  of  pilot  input  increases.  As  the  approach  to  the  boundary 
becomes  more  rapid,  the  magnitude  of  pilot  input  to  avoid  that  boundary  also  increases. 
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Since  this  type  of  tracking  only  occurs  with  pilot  awareness  of  the  imposed 
boundaries,  BAT  is  transient  in  nature.  If  the  boundaries  are  sufficiently  wide  as  to  allow 
a  large  envelope  of  motion,  it  is  quite  possible  that  a  pilot  may  never  engage  in  a  BAT 
event  during  a  specific  task.  However,  as  the  boundaries  become  tighter,  the  pilot- 
aircraft  system  can  become  unstable  and  result  in  PIOs. 

2.2  Modeling  Boundary  Avoidance  Tracking 

In  an  effort  to  better  understand,  explain,  and  characterize  BAT,  Mr.  Gray  created 
the  first  BAT  model  (Gray,  2004: 1 1).  The  system,  shown  in  Figure  2,  was  composed  of  a 
simple  pilot  model  and  a  second  order  system  with  rate  and  displacement  as  the  primary 
feedback  mechanisms.  The  feedback  loop  was  composed  of  both  point  and  boundary 
tracking  algorithms  with  a  logic  switch  that  forced  the  system  to  select  between 
application  of  point  tracking  feedback  gains  or  boundary  tracking  feedback  gains. 


Figure  2.  Diagram  of  Original  Boundary  Avoidance  Tracking  Model 
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2.2.1  Model  Assumptions 

Five  primary  assumptions  were  made  that  governed  the  development  of  Mr. 
Gray’s  original  model  of  boundary  avoidance  tracking.  The  first  was  that  time-to- 
boundary  was  the  critical  parameter  for  boundary  avoidance  tracking  (Gray,  2004:12). 
This  parameter  was  calculated  from  the  system  distance  and  rate  to  the  specified 
boundary. 

boundary  ,  1 

^  boundary  m  v 

X  boundary 

Where: 

^boundary  =  time  to  boundary 

^boundary  =  distance  to  boundary 

x boundary  =  time  rate  of  change  of  distance  to  boundary 

The  use  of  perceived  time  first  showed  promise  during  a  test  of  nuisance  warning 
criteria  in  2001.  Using  an  automated  recovery  system,  pilots  were  placed  in  a  dive 
toward  the  ground.  Once  the  pilot  felt  that  a  recovery  should  occur,  he  activated  the 
automated  recovery  system  and  the  aircraft  recovered  to  a  safe  altitude.  Data  collected 
from  the  test  presented  a  time  available  to  ground  impact,  or  a  time  to  boundary.  The 
times  were  calculated  similarly  to  the  time  to  boundary  presented  here  (Prosser,  2001 :4- 
5). 

The  applied  boundary  tracking  gain  changed  linearly  with  time-to-boundary  and 
varied  between  zero  and  its  specified  maximum.  The  larger  the  time-to-boundary,  the 
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smaller  the  boundary  tracking  gain,  and  vice  versa.  If  the  time-to-boundary  was  greater 
than  the  specified  time-to-boundary  for  minimum  boundary  tracking  gain,  the  boundary 
tracking  gain  was  zero  and  the  system  remained  in  point  tracking.  On  the  other  hand,  if 
the  time-to-boundary  was  less  than  the  specified  time-to-boundary  for  maximum 
boundary  tracking  gain,  it  was  held  at  its  maximum  value. 


K 


t K  min  ^ boundary 


boundary 


K„ 


t 


K  min 


-t 


K  max 


(2) 


Where: 


Kboundary  =  instantaneous  boundary  tracking  feedback  gain 

Kmax  =  maximum  boundary  tracking  feedback  gain 

tKmin  =  time  to  boundary  for  minimum  boundary  tracking  feedback  gain 

tKmax  =  time  to  boundary  for  maximum  boundary  tracking  feedback  gain 

Note.  tKmin  ''  tKmax 


The  second  assumption  was  that  if  the  system  exceeded  a  boundary,  the 
maximum  boundary  feedback  gain  was  held  until  the  system  returned  within  the 
boundaries  (Gray,  2004: 12).  This  assumption  allowed  for  boundary  excursions  to  occur 
and  further  assumed  that  pilots  will  instinctively  maintain  the  maximum  control  input 
possible  to  return  the  system  below  the  specified  boundary  as  quickly  as  possible. 

The  third  assumption  was  that  the  pilot  was  either  entirely  focused  on  the  point 
tracking  task  or  the  boundary  avoidance  task  -  not  both  (Gray,  2004: 12).  As  shown  in 
Figure  2,  the  point  tracking  feedback  loop  was  composed  of  rate  tracking  with  a  simple 
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gain  and  time  delay.  The  model  simultaneously  computed  both  the  boundary  tracking 
feedback  and  the  point  tracking  feedback  and  applied  only  the  one  with  the  greatest 
magnitude. 

The  fourth  assumption  was  that  there  was  no  time  delay  associated  with  the 
switching  between  point  tracking  and  boundary  tracking  (Gray,  2004: 12).  Although 
some  finite  amount  of  time  is  required  for  any  human  to  switch  from  processing  one  set 
of  information  to  another,  this  assumption  provided  a  reasonable  first  step  to  modeling 
boundary  avoidance  tracking.  Furthermore,  any  additional  time  delay  added  to  the 
system  would  only  increase  the  likelihood  of  system  instability  (Gray,  2004:12). 

The  fifth  and  final  assumption  was  that  both  the  boundary  and  point  tracking 
feedback  loops  have  associated  gains  and  maximum  feedbacks  (Gray,  2004: 12)  -  in  other 
words,  each  feedback  loop  is  independent  of  the  other.  Since  pilots  inherently  limit  their 
control  inputs  when  performing  tracking  tasks,  this  assumption  allows  for  more 
accurately  modeling  the  different  types  of  inputs.  When  point  tracking,  pilots  limit  the 
magnitude  of  their  inputs  to  small  deflections  in  an  effort  to  maintain  the  desired 
condition  for  as  long  as  possible.  However,  when  boundaries  are  present  and 
approached,  the  magnitude  of  the  deflections  can  increase  greatly;  especially  if  the 
boundary  is  approached  rapidly  and  is  perceived  as  hazardous  or  life-threatening.  Under 
these  conditions,  pilots  can  command  “stop-to-stop”  deflections  demanding  the 
maximum  performance  from  the  aircraft  system.  Therefore,  by  providing  a  set  of  gains 
and  associated  maximum  feedbacks  for  each  feedback  loop,  the  model  more  accurately 
reflected  the  transition  between  point  and  boundary  tracking. 
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2.2.2  Point  Tracking  Pilot  Model 

A  simple  pilot  model,  composed  solely  of  a  pure  gain  and  time  delay,  was 
implemented  to  simulate  the  point  tracking  portion  of  the  model.  As  shown  in  Figure  2, 
the  displacement  output  was  differentiated  and  sent  into  the  saturation  and  delay  loop  of 
the  point  tracking  feedback  loop.  Saturation  was  added  to  the  system  in  order  to  limit  the 
applied  feedback  gain  (Gray,  2004: 13).  The  resulting  feedback  signal  from  the  point 
tracking  pilot  model  was  then  fed  into  the  feedback  selector. 

2.2.3  Boundary  Tracking  Pilot  Model 

A  similar  pilot  model,  pure  gain  and  time  delay,  was  also  used  for  the  boundary 
tracking  feedback  loop.  However,  the  gain  associated  with  this  loop  was  a  function  of 
the  time-to-boundary  calculated  from  the  displacement  and  rate  to  boundary,  as  stated  in 
equation  2  and  the  first  and  second  model  assumptions.  The  resulting  feedback  signal 
from  the  boundary  tracking  pilot  model  was  then  fed  into  the  feedback  selector. 

The  feedback  selector  compared  the  magnitudes  of  the  point  tracking  and 
boundary  tracking  feedback  loops  and  fed  the  larger  magnitude  into  the  system  plant. 
This  switching  mechanism  corresponds  to  the  third  model  assumption  where  the  pilot  is 
either  totally  involved  with  the  point  tracking  task  or  the  boundary  avoidance  tracking, 
but  not  both.  Overall,  both  the  point  tracking  and  boundary  tracking  pilot  models 
comprise  a  single  switching-type  pilot  model  that  selects  the  gain  necessary  for  the  most 
pressing  tracking  task. 
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2.2.4  System  Plant 

A  simple  second  order  system  was  used  to  provide  representative  aircraft  system 
responses  (Gray,  2004: 13).  The  plant  was  created  with  a  moderate  level  of  damping  to 
simulate  the  oscillatory  response  of  most  aircraft.  To  excite  the  plant,  a  small  pulse  was 
input  into  the  system  at  the  onset  of  the  simulation. 

2.2.5  Model  Results 

The  values  used  by  the  author  in  both  the  pilot  and  plant  models  were  purely 
notional  and  were  selected  to  provide  examples  of  the  different  levels  of  boundary 
avoidance  tracking.  They  do  not  necessarily  represent  any  real  system  past,  present  or 
proposed.  They  do  however  represent  a  system  that  is  driven  to  an  oscillatory  behavior 
as  a  result  of  simply  shrinking  the  distances  between  boundaries.  The  values  selected  for 
the  following  example  are  presented  in  table  1 . 

Table  1.  Boundary  Tracking  Model  Parameter  Values 


Parameter 

Variable 

Value 

Point  Tracking  Gain 

Kpr 

0.58 

Point  Tracking  Gain  Saturation  Limit 

KpT-LIMIT 

0.75 

Point  Tracking  Time  Delay 

Zpp 

0.25  seconds 

Maximum  Boundary  Tracking  Gain 

Kmax 

0.7  i 

Boundary  Tracking  Time  Delay 

Tb 

0.25  seconds 

Time-to-boundary  for  Minimum  Boundary  Tracking  Feedback 

t Kmin 

0.7  seconds 

Time-to-boundary  for  Maximum  Boundary  Tracking  Feedback 

t Kmax 

0.3  seconds 

Boundary  Distance  for  Example  1  (Point  Tracking) 

hb-i 

0.066 

Boundary  Distance  for  Example  2  (Boundary  Awareness) 

hb-2 

0.062 

Boundary  Distance  for  Example  3  (Boundary  Avoidance) 

hb-3 

0.0582 

The  charts  shown  in  Figures  3-5  present  the  results  for  three  different  tracking 
scenarios.  The  first  plot  in  each  of  the  figures  shows  the  feedback  magnitudes  of  both  the 
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point  tracking  and  boundary  tracking  feedback  loops.  The  second  plot  shows  the  input 
signal  and  the  applied  feedback  magnitude.  The  third  and  final  plot  shows  the  time 
history  of  the  tracking  event  -  the  relative  position  of  the  specified  tracking  task  to  the 
target  condition  and  the  position  with  respect  to  the  applied  boundaries.  The  scales  for 
both  feedback  magnitudes  and  relative  positions  are  not  included  or  required  for  this 
preliminary  analysis  as  the  difference  in  tracking  events  are  the  focus.  Finally,  all  applied 
boundaries  were  symmetric  in  nature;  that  is  they  were  the  same  magnitude  applied 
symmetrically  about  the  point  tracking  target. 

The  first  example  was  a  case  of  pure  point  tracking.  By  setting  the  boundaries  to 
a  relatively  large  distance  (hb-i),  the  pilot  never  perceives,  or  becomes  aware  of,  the 
imposed  boundaries.  As  shown  in  Figure  3,  the  magnitude  of  the  boundary  feedback 
remains  zero  throughout  the  event.  The  system  continues  to  oscillate  about  the  desired 
point  tracking  condition  and  shows  the  typical  exponential  decay  of  a  second-order 
system. 
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Figure  3.  Boundary  Tracking  Model  Result  -  Point  Tracking 

The  second  example,  Figure  4,  is  of  boundary  awareness.  The  only  model 
parameter  changed  here  was  a  relatively  small  decrease  in  the  boundary  distance  (hb-i 
changed  to  hb-i)-  Here  the  relatively  closer  boundaries  cause  the  pilot  to  momentarily 
switch  to  boundary  avoidance  tracking  to  ensure  that  the  boundary  has  been  adequately 
avoided.  Once  the  boundary  has  been  successfully  avoided,  the  pilot  returns  to  the  point 
tracking  task  and  continues  to  input  to  minimize  the  error  associated  with  the  target 
condition. 
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Figure  4.  Boundary  Tracking  Model  Result  -  Boundary  Awareness 

The  transient  input  associated  with  the  boundary  is  in  fact  a  boundary  avoidance 
event.  What’s  important  with  this  example  is  that  boundary  awareness  has  occurred. 

That  is,  an  input  was  made  into  the  system  in  an  effort  to  avoid  an  established  boundary. 
Boundary  awareness  must  occur  for  boundary  avoidance  to  take  place  and  awareness  is 
governed  by  the  time-to-boundary  for  minimum  boundary  tracking  feedback  gain, 

The  final  example,  Figure  5,  is  of  multiple  boundary  avoidance  events.  Again  the 
only  parameter  adjusted  for  this  example  was  another  relatively  small  decrease  in  the 
boundary  distance  ( hb-2  changed  to  hhs).  Here  the  system  oscillations  are  not  solely 
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caused  by  the  small  oscillations  of  the  second-order  response,  but  by  repeated  inputs 
resulting  from  boundary  avoidance.  The  system  eventually  stabilizes  as  in  the  previous 
two  examples  and  returns  to  point  tracking  only.  However,  larger  amplitude  oscillations 
are  initially  sustained  in  the  system  response  due  to  the  elevated  gains  associated  with  the 
boundary  avoidance  events. 

From  this  example,  it  is  apparent  that  when  boundaries  are  placed  sufficiently 
close  to  the  primary  point  tracking  task,  the  system  can  oscillate  more  frequently  and  for 
longer  durations  than  with  an  unbounded  point  tracking  task.  In  fact,  for  the  model 
values  presented  here,  any  further  decrease  in  boundary  distance  results  in  maximum 
boundary  tracking  feedback  and  system  instability  with  divergent  system  oscillations. 


18 


I - 1 - 1 - 1 - 1 - 1 - T - T - T - [~ 

0  0.5  1  1.5  2  2.5  3  3.5  4  4.5 

Time 


Figure  5.  Boundary  Tracking  Model  Result  -  Boundary  Avoidance  Tracking 


2.2.6  Boundary  Avoidance  Tracking  Modeling  Summary 

Mr.  Gray’s  initial  study  proved  the  ability  to  successfully  model  the  BAT  theory 
and  provided  a  baseline  for  identifying  the  key  parameters  that  characterize  boundary 
avoidance  tracking.  Holding  all  parameters  constant,  it  is  apparent  that  increasing  the 
stringency  of  imposed  boundaries  can  lead  to  system  instabilities  and  unfavorable  aircraft 
handling  qualities.  It  is  essential,  therefore,  that  BAT  be  more  fully  investigated  and  the 
theory  developed  to  better  aid  the  aerospace  community  in  improving  aircraft  handling 
qualities  prediction  and  testing. 
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The  initial  model  also  provided  a  roadmap  for  the  continued  study  of  boundary 
tracking.  As  discussed  previously,  all  the  parameters  associated  with  the  pilot  models 
and  the  system  plant  were  variable  and  not  necessarily  representative  of  past,  present,  or 
proposed  systems.  The  next  steps  for  investigating  boundary  avoidance  tracking  are  then 
to  determine  representative  values  of  the  parameters  that  characterize  boundary 
avoidance  tracking  (fenm,  tKmax,  and  Kmax)  and  to  investigate  the  relationship  between 
those  parameters  and  the  parameters  that  define  the  system  plant,  the  flying  qualities. 

The  aircraft  flying  qualities  are  indicators  of  the  ability  of  the  closed-loop  pilot 
plus  aircraft  system  to  effectively  conduct  a  specified  task.  If  a  relationship  between 
these  parameters  and  the  parameters  that  define  BAT  can  be  obtained,  then  a  better 
understanding  of  the  effect  that  boundary  avoidance  has  on  pilot  tracking  will  result.  The 
developed  model  has  already  indicated  that  the  mere  presence  of  boundaries  affects  the 
stability  of  a  closed-loop  system. 

2.3  Simulating  Boundary  Avoidance  Tracking 

Taking  the  next  step,  Mr.  Gray  developed  a  pilot-in-the-loop  simulation  utilizing 
the  flying  qualities  of  a  North  American  Navion  (Gray,  2005:6).  Developed  using 
Simulink®,  the  simulator  presented  a  bounded  tracking  task  to  the  pilot  and  recorded  the 
instantaneous  stick  position  for  analysis.  Eight  subjects  of  varying  flight  experience  each 
spent  approximately  30  minutes  in  the  simulator. 

2.3.1  Simulator  Tracking  Task 

The  developed  task  was  to  match  the  altitude  of  a  lead  Navion  that  was  constantly 
changing  altitude  through  small  amplitude  inputs  (Gray,  2005:6).  The  test  subjects  were 
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provided  with  a  simple  display  that  provided  the  relative  altitude  of  the  pilot’s  aircraft 
with  respect  to  the  lead  Navion  (Figure  6).  The  goal  was  to  minimize  the  altitude  error 
between  the  two  aircraft  by  maintaining  the  aircraft  altitude  line  on  the  zero  reference  and 
simultaneously  avoiding  the  boundaries. 


Figure  6.  Gray  Simulator  Tracking  Task  Display 


The  boundaries  were  brought  25%  closer  to  the  zero  reference  every  minute  and 
the  oscillatory  flight  path  of  the  lead  Navion  was  also  repeated  every  minute.  Each 
subject  flew  three  different  scenarios:  first,  the  standard  Navion  tracking  task;  second,  a 
300  millisecond  time  delay  added  to  the  pilot’s  inputs;  and  third,  a  0.3  radian  per  second 
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rate  limit  placed  on  the  deflection  of  the  horizontal  tail  (Gray,  2005:6).  The  simulation 
was  stopped  once  a  boundary  was  crossed. 

2.3.2  Simulator  Data  Reduction 

All  data  from  the  pilot-in-the-loop  simulation  was  collected  at  a  rate  of  100 
samples  per  second.  The  key  boundary  tracking  parameters  (Am,-„,  tKmax,  Kmax)  were 
determined  by  applying  the  previously  developed  boundary  tracking  model  to  the 
collected  simulator  results.  The  key  parameters,  along  with  the  pilot  time  delay,  were 
varied  until  the  model  results  closely  matched  the  simulator  data. 

2.3.3  Simulator  Results 

A  total  of  27  boundary  avoidance  events  were  evaluated.  For  events  that  lasted 
60  seconds  or  longer,  the  values  of  A,„,«  and  tKmax  were  both  found  to  possess  a  strong 
central  tendency.  For  tKmin,  the  mean  was  2.8  seconds  with  a  variance  of  0.25  seconds. 
The  mean  for  tKmax  was  0.3  seconds  with  a  variance  of  0.04  seconds  (Gray,  2005: 1 1). 

The  maximum  boundary  tracking  feedback  gain  was  found  to  correlate  well  with 
the  test  subject’s  ability  to  track  for  a  long  duration.  The  most  interesting  results  were 
provided  from  a  developed  parameter,  termed  the  boundary  tracking  parameter  (BTP). 


BTP  = 


min  max 

K _ 


(3) 


This  parameter  showed  some  potential  correlation  with  the  length  of  successful 
tracking  time.  As  the  BTP  value  increased,  so  did  the  duration  of  the  successful  tracking 
time  (Figure  7).  This  result  makes  intuitive  sense  in  that  as  pilots  are  able  to  limit  the 
magnitude  of  their  inputs,  and  their  associated  gains,  they  are  able  to  maintain  system 
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stability  longer  and  successfully  perfonn  the  tracking  task  for  a  longer  duration.  Also, 
while  holding  the  pilot  gain  constant,  as  the  difference  between  tKmin  and  tKmax  decreases, 
so  too  does  the  duration  of  the  successful  tracking  time.  This  makes  sense  in  that  if  the 
difference  between  these  two  parameters  is  too  small,  the  pilot  will  then  rapidly  transition 
to  a  maximum  input  and  cause  the  system  to  become  unstable  more  rapidly  (Gray, 
2005:11). 
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Figure  7.  Boundary  Tracking  Parameter  (BTP)  Results  (Gray,  2005:11) 


Overall,  the  results  of  Mr.  Gray’s  simulator  study  provided  for  an  initial 
characterization  of  BAT.  Perception  of  time-to-boundary  appears  to  be  an  inherent 
human  characteristic  and  the  more  experienced  and  less  aggressive  the  pilot,  the  more 
successful  the  ability  to  track  in  the  presence  of  boundaries.  These  results  lay  the 
foundation  for  the  research  that  was  conducted  for  this  thesis. 


23 


3.  Research  Objectives  and  Experiment  Methodology 


3.1  Research  Overview 

3.1.1  System  Definition 

In  order  to  properly  evaluate  the  effect  of  BAT,  it  is  important  to  begin  by 
defining  the  system  that  was  investigated.  The  system  under  experimentation  for  this 
research  was  the  complex  pilot  plus  aircraft  system.  Termed  “the  pilot’s-eye  system”, 
this  system  comprised  all  the  steps  between  the  pilot’s  perception  of  an  impending 
boundary  and  the  aircraft  response  from  the  pilot’s  system  inputs.  Significant  parts  of  the 
system  included  the  pilot’s  eye,  brain,  and  hands,  the  aircraft  control  stick’s  tactile 
feedback  mechanisms,  the  flight  control  system,  and  the  aircraft  aerodynamics.  Use  of 
this  system  was  critical  to  characterizing  the  time-to-boundary  parameters  as  they  are 
functions  of  pilot  perception  and  response. 

In  order  to  faithfully  represent  the  pilot’s  system  feedback,  a  video  camera  system 
was  used  to  capture  the  pilot’s  perspective  of  the  tracking  task  as  closely  as  possible. 

Data  recorders  were  also  used  to  capture  the  pilot’s  system  inputs. 

The  T-38C  aircraft,  originally  produced  by  Northrop  Corporation,  Aircraft 
Division  and  upgraded  by  The  Boeing  Company,  is  a  two-place,  twin-turbojet  supersonic 
trainer.  The  aircraft  is  equipped  with  an  all-movable  horizontal  tail  and  a  hydraulically 
powered,  irreversible  flight  control  system.  The  control  surfaces  are  non-reversible,  so 
conventional  aerodynamic  feel  in  the  control  stick  was  provided  artificially  by  springs 
and  bobweights. 
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3.1.2  Boundary  Avoidance  Tracking  Parameters 

As  stated  in  Chapter  2,  time-to-boundary  was  assumed  to  be  the  critical  parameter 
in  characterizing  BAT.  The  key  times-to-boundary  are  Ik  mm,  for  the  time-to-boundary  at 
minimum  boundary  tracking  gain  and  tKmax,  for  the  time-to-boundary  at  the  maximum 
BAT  gain.  Physically,  the  parameter  tKmm  corresponds  to  the  perceived  time-to-boundary 
at  which  a  pilot  first  responds  to  the  boundary  by  making  an  input  into  the  system.  The 
tKmax  parameter  then  refers  to  the  time-to-boundary  at  which  the  pilot  applies  a  maximum 
system  input  to  avoid  the  impending  boundary. 

The  BAT  gain,  which  is  the  magnitude  of  the  system  input  by  the  pilot,  is  a  result 
of  the  pilot’s  perceived  time-to-boundary.  As  shown  in  equation  2,  the  BAT  gain  was 
assumed  to  vary  linearly  between  tKmin  and  tKmax  from  zero  to  a  maximum  value. 
Therefore,  it  is  the  maximum  BAT  gain,  Kmax,  that  is  of  interest  in  helping  to  characterize 
BAT  events. 

The  boundary  tracking  parameter  (BTP),  which  combines  all  three  parameters  into 
a  single  parameter,  was  also  of  interest.  It  was  thought  that  this  parameter  acted  as  a 
nonnalizing  agent.  The  results  of  the  research  did  not  confirm  this  assumption.  More 
experienced  pilots  will  typically  allow  themselves  to  react  to  a  boundary  at  a  smaller 
time-to-boundary  than  less  experienced  pilots.  Likewise,  more  experienced  pilots  will 
limit  their  system  inputs  to  smaller  magnitudes  while  neophytes  will  typically  use  much 
larger  inputs. 

Finally,  of  interest  was  the  frequency  at  which  the  pilot  moved  the  longitudinal 
stick  during  BAT  events.  This  pilot  actuation  frequency,  cobat,  would  be  used  for 
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frequency  domain  analysis  of  the  system  to  detennine  how  closely  the  pilots  were 
operating  to  the  system  crossover  frequency  -  the  frequency  at  which  the  system  has  a 
phase  of  -180°.  Applying  the  definition  -  “A  PIO  exists  when  the  airplane  attitude, 
angular  rate,  or  normal  acceleration  is  180  degrees  out  of  phase  with  the  pilot’s  control 
inputs”  (Mitchell  and  Hoh,  1995:18)  -  the  airplane  attitude  can  be  considered  analogous 
to  the  pilot’s  perceived  relative  position.  As  such,  comparison  of  cobat  with  the  system 
crossover  frequency  will  provide  a  result  of  system  PIO  sensitivity.  Once  all  of  the  BAT 
parameters  had  been  determined  and  characterized,  correlating  them  with  the  aircraft 
flying  qualities  could  be  carried  out. 

3.1,3  Aircraft  Flying  Qualities 

Flying  qualities  are  the  parameters  that  characterize  the  dynamic  behavior  of  a 
piloted  aircraft.  Through  several  decades  of  research  and  flight  test,  countless  parameters 
have  been  developed  to  better  understand,  predict,  and  model  aircraft  flying  qualities  in 
an  effort  to  provide  aircraft  that  exhibit  desirable  characteristics.  Though  initial  research 
into  boundary  avoidance  tracking  provide  for  a  single  time-to-boundary  for  boundary 
awareness  and  a  single  value  of  time-to-boundary  for  maximum  boundary  feedback  gain 
across  a  wide  range  of  pilot  experiences,  the  effect  that  the  aircraft’s  flying  qualities  have 
on  the  boundary  tracking  parameters  has  never  been  investigated.  Finding  a  correlation 
or  an  independence  between  these  parameters  will  ultimately  lead  to  better 
characterization  and  understanding  of  BAT  and  the  effect  it  has  on  pilot  tracking  success. 

As  a  first  step  into  the  investigation  of  the  relationship  between  the  BAT 
parameters  and  flying  qualities,  two  parameters  were  selected  based  on  Mr.  Gray’s 
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previous  work  and  based  on  classical  systems  analysis.  The  first  parameter  was  the 
system  time  delay,  t, delay  -  the  difference  in  time  from  the  system  input  to  associated 
system  motion.  Excessive  system  time  delay  is  known  to  be  a  contributor  to  poor  aircraft 
handling  qualities  and  was  identified  in  Mr.  Gray’s  simulator  study  as  a  contributing 
factor  to  decreased  successful  tracking  times  (Gray,  2005:1 1). 

The  second  parameter  was  the  system  crossover  frequency,  com  -  the  frequency 
at  which  system  input  is  180  degrees  out  of  phase  with  system  output.  This  parameter 
was  of  interest  in  comparison  with  the  pilot  actuation  frequency,  (Qbat-  If  the  pilots  were 
operating  at  or  near  the  crossover  frequency  during  BAT  events,  the  system  could  be 
unstable  and  lead  to  undesirable  aircraft  oscillations. 

All  of  the  research  parameters  of  interest  have  been  defined  for  the  pilot’ s-eye 
system  of  interest.  The  parameters,  tKmin-  TKmax,  Kmax,  BTP,  and  cobat  will  be  determined 
and  analyzed  in  order  to  characterize  BAT.  Then  the  aircraft  flying  qualities  parameters, 
Tdeiay  and  co i so,  will  be  detennined  and  correlated  with  the  BAT  parameters  to  see  what 
the  effect  is  on  pilot  tracking  success. 

3.2  Experiment  Design 

The  techniques  developed  to  collect  the  parameters  of  interest  were  centered 
around  the  specified  point  tracking  task.  In  this  section,  the  details  of  that  tracking  task, 
the  mindset  of  the  pilots  during  the  tracking  task,  and  the  conditions  for  the  experiment 
will  be  discussed. 
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3.2.1  Pilot  Tracking  Task 

The  point  tracking  task  was  simple:  fly  a  specific  formation  position  off  the  target 
aircraft  by  tracking  reference  markings  and  maintain  the  target  aircraft’s  wingtip  within 
the  specified  boundaries.  The  reference  markings  for  the  point  tracking  task  were  to 
maintain  the  front  corner  of  the  wingtip  of  the  target  aircraft  level  with  the  star  in  the 
chevron  on  the  fuselage  (see  Figure  8).  The  reference  markings  for  the  boundaries  were 
the  solid  and  dashed  lines  immediately  above  and  below  the  chevron.  The  basic 
formation  position  for  the  experiment  approximated  a  “route”  position  with 
approximately  ten  foot  wingtip  spacing.  The  pilot  continued  the  point  tracking  task 
throughout  the  test  point,  abandoning  it  only  when  boundary  penetration  was  perceived  to 
become  a  threat,  or  the  test  point  was  terminated  (if  the  tracking  task  could  be  continued 
without  any  boundary  avoidance  tracking  or  due  to  exceeding  the  databand). 


Figure  8.  Tracking  Task  Visual  Reference 


28 


3.2.2  Pilot  Mindset 


Because  of  the  inescapable  human  element  of  the  experiment,  the  individual  pilots 
could  possibly  generate  different  results  while  performing  the  tracking  tasks  under 
similar  conditions.  Control  aggressiveness,  reaction  speed,  and  self-limitation  (self- 
imposed  limits  to  avoid  over-g  or  excessive  buffet)  could  potentially  vary  widely  from 
pilot  to  pilot.  Because  the  pilot  was  a  key  factor  in  the  experiment,  tracking  task  data 
were  obtained  from  all  of  the  pilots  at  each  test  condition. 

In  an  attempt  to  faithfully  recreate  the  stress  of  a  pilot  who  perceives  an  oncoming 
hazard,  the  test  pilots  flew  with  a  specific  mindset:  allowing  the  wingtip  to  deviate 
beyond  the  imposed  horizontal  boundary  markings  was  unacceptable.  With  real  aircraft 
structural  and  operational  limits  in  mind,  the  pilots  would  naturally  limit  the  magnitude  of 
their  inputs.  To  alleviate  this  potential  additional  boundary,  the  experiments  were 
designed  to  remove  or  reduce  the  risk  of  exceeding  load  factor  limits.  This  allowed  the 
pilots  to  maneuver  with  minimal  restraint. 

3.2.3  Flight  Conditions 

Without  a  variable  stability  aircraft,  the  flight  conditions  of  the  experiment  had  to 
be  varied  in  order  to  provide  a  range  of  aircraft  flying  qualities  for  analysis.  The 
aerodynamic  effects  of  the  system  were  altered  by  performing  the  flight  test  techniques  at 
different  altitudes,  airspeeds,  and  landing  gear  configurations  to  cause  differences  in  the 
aircraft  flying  qualities.  The  flaps  and  speed  brakes  were  not  extended  for  any  test 
points.  Table  2  lists  the  test  conditions  used  for  this  experiment. 
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Table  2.  Summary  of  Flight  Conditions 


Flight  Condition 

kltitude 

(feet) 

Landing  Gear 
Position 

Airspeed 

(KIAS) 

Typical 

Mach 

High  Fast  Cruise 

20,000 

Up 

375 

0.80 

High  Slow  Cruise 

20,000 

Up 

250 

0.55 

High  Configured 

20,000 

Down 

210 

0.46 

Low  Fast  Cruise 

10,000 

Up 

375 

0.67 

Low  Slow  Cruise 

10,000 

Up 

250 

0.45 

Low  Configured 

10,000 

Down 

210 

0.38 

3.3  Fixed-Base  Simulation 

In  order  to  properly  prepare  for  the  experiment,  a  simulator  was  used  by  the  flight 
test  team  to  develop  flight  test  techniques  for  both  aircraft  flying  qualities  and  boundary 
avoidance  tracking.  The  simulator  was  developed  by  Lieutenant  Jay  Kemper  of  the  Air 
Force  Research  Laboratory  Air  Vehicles  Directorate  (AFRL/VA)  over  a  one  year  period. 

3.3.1  Simulator  Properties 

The  T-38A  simulator  was  provided  by  the  Air  Force  Aeronautical  Systems  Center 
(ASC)  Training  Aircraft  Group  T-38  Systems  Squadron.  The  simulator  was  a  full  non¬ 
linear  six  degree  of  freedom  simulation  of  a  T-38  A  aircraft  programmed  in  the  ADA 
programming  language. 

Unfortunately  the  AFRL/VA  simulators  were  not  able  to  utilize  the  ADA 
simulation.  Lieutenant  Kemper  developed  both  a  modified  version  of  the  ADA-based 
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simulation  as  well  as  a  communications  algorithm  that  allowed  the  AFRL/VA  simulators 
to  successfully  execute  the  T-38A  simulation.  This  effort  was  crucial  to  the  success  of 
both  a  highly  functional  and  high  fidelity  T-38A  simulator  and  the  development  of 
experiment  specific  flight  test  techniques. 

The  Infinity  Cube  simulator  provided  a  full  180  degrees  of  visibility  with  full  look¬ 
up  capability  through  the  use  of  four  independent  video  projectors.  The  aircraft  inceptors 
were,  however,  not  indicative  of  the  controls  found  in  a  T-38  aircraft.  The  simulator  was 
configured  for  simulation  of  F-16  aircraft  with  a  force-feedback  side  stick  and  a  single 
throttle  control.  As  T-38  aircraft  use  a  movable  center  stick  and  two  throttle  controllers, 
the  cockpit  inceptors  were  not  representative  of  what  the  flight  test  team  would  be  using 
during  the  experiment.  Furthermore,  the  graphical  display  of  the  simulator  could  not  be 
altered  to  provide  representative  visual  boundaries  that  would  be  used  during  the 
experiment. 
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Figure  9.  AFRL  Simulator  Visual  Presentation  (Left  Panel  Display) 


Overall,  though,  the  test  pilot  flight  crew  members  reported  that  the  aircraft 
response  and  handling  qualities  were  good  representations  of  actual  T-38A  aircraft.  The 
visual  presentations  (Figure  9)  provided  the  flight  crew  members  with  an  excellent 
representation  of  the  flight  conditions  that  would  be  experienced  during  the  execution  of 
the  experiment.  For  the  use  of  flight  test  technique  development,  the  AFRL/VA 
developed  T-38A  simulator  was  an  excellent  resource  that  saved  valuable  experiment 
flight  time. 


32 


3.3.2  Aircraft  Flying  Qualities  Flight  Test  Technique  Development 

Utilizing  the  pilot’ s-eye  system,  flying  qualities  flight  test  techniques  had  to  be 
developed  that  were  indicative  of  the  pilot  plus  aircraft  open-loop  system.  For  the  system 
time  delay  parameter,  the  technique  selected  was  a  traditional  time  delay  technique  - 
application  of  a  step  input  followed  by  measurement  of  the  delay  from  the  time  of  the 
input  to  the  first  aircraft  response.  Trial  of  this  technique  in  the  simulator  showed  the 
maneuver  was  quick  and  easy  to  replicate.  A  few  iterations  of  this  technique  by  each  test 
pilot  provided  confidence  in  the  execution  of  the  maneuver  and  gave  the  test  team  a 
better  feel  for  the  time  required  for  test  point  set-up,  execution  and  the  maneuvers 
required  for  reestablishing  the  formation. 

To  capture  the  system  crossover  frequency,  the  traditional  frequency  sweep  flight 
test  technique  was  modified  to  reflect  the  difference  in  traditional  aircraft  systems  and  the 
pilot’ s-eye  system.  In  traditional  frequency  sweeps,  the  pilot  slowly  moves  the  aircraft 
stick  forward  and  aft  to  induce  aircraft  motion.  The  pilot  then  begins  to  increase  the 
frequency  of  the  forward  and  aft  inputs  so  that  a  wide  range  of  frequencies  are  covered 
and  frequency  domain  analysis  tools  can  be  applied. 

For  the  pilot’ s-eye  system  used  in  the  experiment,  the  frequency  sweep  inputs 
were  similar  to  those  as  described  in  the  traditional  frequency  sweep.  However,  the 
system  output  was  defined  as  the  relative  altitude  difference  between  the  target  aircraft 
and  the  test  aircraft.  So,  pilot  inputs  were  applied  with  the  traditional  technique  and  the 
system  output  measured  from  the  visual  data  captured  by  the  camera  system. 
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3.3.3  Boundary  Avoidance  Tracking  Flight  Test  Technique  Development 

As  described  in  the  pilot  task  section,  the  goal  of  the  pilot  was  to  maintain  the 
target  aircraft  wingtip  in  the  center  of  the  chevron  and  prevent  the  wingtip  from  crossing 
any  imposed  boundary.  The  development  of  a  mechanism  to  provide  an  approach  to 
boundaries  was  also  necessary  to  excite  potential  BAT  events.  As  such,  two  methods 
were  chosen  -  decreasing  boundary  distances  and  target  aircraft  maneuvering. 

The  decreasing  boundary  method  required  that  the  target  aircraft  remain  at  the 
specified  flight  condition  and  configuration  while  the  test  aircraft  gradually  decreased  the 
boundary  distances.  To  simulate  this,  the  test  pilots  utilized  the  top  and  bottom  of  the 
target  aircraft’s  fuselage  just  above  and  below  the  chevron  (see  Figure  9)  as  well  as  the 
top  and  bottom  of  the  blue  circle  in  the  chevron.  All  pilots  found  it  relatively  easy  to 
maintain  the  desired  formation  position  even  as  the  boundary  distances  were  decreased 
during  the  simulations.  Though  this  method  rarely  produced  a  BAT  event  or  aircraft 
oscillation  in  the  simulation,  it  was  adopted  for  use  in  the  experiment  as  a  baseline 
maneuver.  This  maneuver  was  termed  the  stable  platform  maneuver. 

The  target  aircraft  maneuvering  methods  provided  the  best  results  in  the  simulator 
for  BAT  events.  The  overall  premise  for  these  types  of  maneuvers  was  to  have  the  target 
aircraft  perform  a  maneuver  that  would  create  a  perceived  approach  to  a  boundary  by  the 
test  pilot.  Three  separate  maneuvers  were  developed  in  the  simulator.  The  first 
maneuver,  tenned  the  pull-up,  was  a  simple  method  for  creating  an  approach  to  the 
boundary  located  below  the  target  chevron.  The  pull-up  required  the  target  aircraft  to 
pull  to  1.5  Gs  within  2  seconds  and  maintain  the  pitch  attitude.  This  maneuver  caused  the 
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target  aircraft  to  begin  a  climb  which  the  pilot  of  the  test  aircraft  perceived  as  an 
approach  to  the  lower  boundary. 

The  second  maneuver,  tenned  the  push-over,  was  similar  to  the  pull-up,  except  the 
target  aircraft  performed  a  push  to  0.5  G  within  2  seconds  and  maintained  the  pitch 
attitude.  The  push-over  created  an  approach  to  the  boundary  located  above  the  target 
chevron.  Both  the  pull-up  and  push-over  maneuvers  showed  great  success  in  the 
simulator  and  were  very  easy  to  set-up  and  execute.  The  test  pilots  again  utilized  the  top 
and  bottom  of  the  fuselage  and  the  top  and  bottom  of  the  blue  circle  in  the  chevron  as 
boundaries  during  the  simulation  producing  repeated  BAT  events. 

The  third  maneuver,  termed  the  rollercoaster,  was  a  combination  of  both  the  pull- 
up  and  the  push-over  and  was  derived  from  the  simulator  study  conducted  by  Mr.  Gray 
(Gray,  2005:6).  The  rollercoaster  had  the  target  aircraft  begin  from  steady  level  flight 
and  then  perform  a  series  of  pull-ups  and  pushovers.  The  series  began  from  1  G  with  a 
±0. 1  G  pull-up/push-over  and  then  increasing  by  ±0. 1  G  to  a  maximum  pull-up  of  1 .5  G 
and  a  minimum  push-over  to  0.5  G.  No  G  rate  was  specified  for  the  maneuver.  Once 
that  level  was  reached,  the  series  then  reversed  and  gradually  returned  to  1  G.  This 
maneuver  always  produced  BAT  events  and  divergent  oscillations  by  all  pilots  for  all 
boundaries  in  the  simulator.  This  was  by  far  the  most  dynamic  of  the  target  aircraft 
maneuvering  methods  evaluated  in  the  Infinity  Cube  simulation.  Table  3  below 
summarizes  all  of  the  flight  test  techniques  developed  for  the  experiment. 
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Table  3.  Summary  of  Developed  Flight  Test  Techniques 


Maneuver 

Area 

Description 

System  Time  Delay 

Flying  Qualities 

Step  input  until  aircraft  motion  was  evident; 
measured  time  lag  from  input  to  aircraft 
response 

Frequency  Sweep 

Flying  Qualities 

Increasing  frequency  of  longitudinal  inputs; 
system  response  measured  visually  as  change 
in  relative  altitude 

Stable  Platfonn 

BAT 

Target  aircraft  remained  stable  and  non¬ 
maneuvering;  test  aircraft  gradually 
decreased  boundaries  until  smallest 
boundaries  reached  or  boundary  excursion 
occurred 

Pull-Up 

BAT 

Target  aircraft  perfonned  pull-up  to  1.5  G 
within  2  seconds;  test  aircraft  perceived 
approach  to  lower  boundary  and  responded 

Push-Over 

BAT 

Target  aircraft  performed  push-over  to  0.5  G 
within  2  seconds;  test  aircraft  perceived 
approach  to  upper  boundary  and  responded 

Rollercoaster 

BAT 

Target  aircraft  performed  varying  pull-up  and 
push-over  maneuvers;  test  aircraft  perceived 
repeated  approaches  to  both  upper  and  lower 
boundaries 

3.4  Experiment  Instrumentation 

The  target  aircraft  used  for  the  experiment  were  marked  with  two  types  of  decals 
for  both  pilot  tracking  and  for  post-flight  data  analysis.  The  test  aircraft  was  equipped 
with  both  a  camera  system  for  recording  a  pilot’ s-eye  representative  visual  image  of  each 
event  as  well  as  an  integrated  data  acquisition  system  to  capture  aircraft  stick 
displacement.  Each  item  is  detailed  in  the  following  sections. 
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3.4.1  Aircraft  Visual  Markings 

Two  types  of  visual  markings  were  required  on  the  target  aircraft  for  the 
experiment.  The  first  were  boundary  lines  that  would  be  referenced  by  the  pilot  of  the 
test  aircraft  for  each  of  the  maneuvers.  The  second  type  was  target  markings  designed  to 
aid  an  automated  visual  tracking  algorithm.  These  markings  are  shown  in  Figure  10 
which  depicts  the  target  aircraft  during  an  experiment  flight. 


Figure  10.  Target  Aircraft  Visual  Markings 

The  first  type  of  markings,  the  boundary  lines,  are  shown  in  Figure  10  as  solid 
and  dashed  black  lines  located  on  the  fuselage  and  travel  pod  of  the  target  aircraft.  These 
lines  were  the  boundaries  that  the  pilot  of  the  test  aircraft  used  during  each  of  the 
maneuvers.  A  third  set  of  boundaries  was  also  established  if  smaller  boundary  distances 
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were  required.  These  boundaries  were  the  top  and  bottom  of  the  circle  in  the  chevron  on 
the  target  aircraft,  the  same  boundaries  previously  established  in  the  fixed-based 
simulator. 

The  second  type  of  markings,  the  video  tracking  software  markings,  were 
specially  designed  for  use  with  an  automated  video  tracking  algorithm.  These  red  and 
yellow  series  of  dots  would  be  automatically  located  in  each  video  frame  allowing  for 
boundary  position  information  to  be  provided  for  further  data  analysis.  Since  the  wingtip 
position  was  also  required  for  analysis,  a  green  decal  was  used  to  allow  the  video  tracking 
algorithm  to  autonomously  track  its  position.  As  can  be  seen  in  Figure  10,  the  wingtip  on 
the  T-38A  was  extremely  thin  and  difficult  to  see  even  with  a  bright  green  decal.  This 
led  to  severe  problems  with  post- flight  data  reduction  which  is  discussed  in  detail  in 
Chapter  4. 

3.4.2  Aircraft  Mounted  Camera  System 

To  capture  the  visual  stimuli  of  the  pilot’s-eye  system  as  closely  as  possible,  a 
camera  system  was  installed  immediately  behind  the  ejection  seat  of  the  front  cockpit. 
This  aircraft  mounted  camera  system  was  approximately  three  feet  behind  and  slightly 
higher  than  the  eyes  of  the  pilot,  as  shown  in  Figure  1 1 . 

The  camera  was  fixed  into  position  by  a  bracket  so  some  amount  of  trial  and  error 
was  required  to  properly  align  the  camera’s  field  of  view;  however,  once  calibrated  the 
camera  required  no  maintenance  or  recalibration  throughout  the  testing.  The  installation 
of  the  aircraft-mounted  camera  was  a  minor  modification  that  integrated  the  camera  with 
the  aircraft  electrical  system  for  power  and  with  three  8mm  tape  decks  in  the  rear  cockpit. 
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One  tape  recorded  the  production  aircraft  HUD/MFD  feed,  the  second  recorded  the 
aircraft-mounted  camera  feed,  and  the  third  recorded  the  data  acquisition  system  data. 
All  three  tapes  contained  a  time  stamp  for  data  synchronization.  The  video  frame  rate 
was  30  frames  per  second. 


Figure  11.  Aircraft  Mounted  Camera  Placement 

A  helmet-mounted  camera  system  was  used  on  every  flight  during  the  experiment 
in  an  effort  to  more  closely  capture  what  the  pilot  was  viewing  during  each  event.  The 
system  was  comprised  of  a  small  lipstick-style  camera  attached  to  the  pilot’s  helmet,  a 
video  tape  recorder,  and  batteries  which  were  self-contained  in  a  modified  survival  vest 
and  worn  by  the  pilot  during  each  flight.  Unfortunately,  the  system  rarely  captured  the 
target  aircraft  fully  in  the  field  of  view  despite  repeated  calibrations  and  adjustments. 
Though  not  directly  at  the  pilot’s  point-of-view,  the  video  obtained  from  the  aircraft- 
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mounted  camera  system  was  a  very  close  representation  of  what  the  pilots  viewed  and 
was  an  excellent  substitute  for  a  helmet-mounted  system. 

3.4.3  Aircraft  Data  Acquisition  System 

The  aircraft  data  acquisition  system  (DAS)  recorded  multiple  aircraft  parameters 
during  each  flight.  However,  only  four  parameters  were  needed  for  this  experiment  - 
longitudinal  stick  position,  aircraft  load  factor  at  the  pilot’s  location,  indicated  airspeed, 
and  aircraft  pitch  attitude.  All  data  parameters  were  recorded  at  a  rate  of  20  samples  per 
second  with  the  longitudinal  stick  position  data  provided  in  units  of  degrees,  load  factor 
in  Gs,  indicated  airspeed  in  knots,  and  pitch  in  degrees. 
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4.  Analysis  and  Results 


4.1  Research  Assumptions 

Before  going  into  the  analyses  and  results  of  the  collected  data,  it  is  important  to 
introduce  two  primary  assumptions  that  governed  both  the  reduction  of  the  data  and  the 
analyses  resulting  from  the  data  reduction. 

4.1.1  Aircraft  Mounted  Camera 

The  aircraft  mounted  camera  system  was  assumed  to  be  representative  of  the  pilot’s 
viewpoint.  Comparison  of  the  collected  video  footage  from  a  helmet  mounted  camera 
and  the  aircraft  mounted  camera  showed  that  the  differences  between  the  two  viewpoints 
were  minor.  This  was  also  confirmed  by  the  test  pilots  after  their  review  of  the  aircraft 
mounted  camera  footage. 

This  assumption  allowed  for  the  use  of  the  visual  data  collected  by  the  aircraft 
mounted  camera  system  to  represent  the  error  signal  in  the  pilot’s-eye  system.  The 
aircraft  mounted  camera  footage  was  more  reliable  than  the  helmet  mounted  camera  and 
also  provided  a  much  more  stable  image.  Even  though  the  aircraft  mounted  camera  was 
placed  approximately  three  feet  behind  the  pilot’s  head  ,  this  system  provided  accurate 
and  reliable  visual  data  for  analysis  of  BAT  events  in  the  pilot’s-eye  system. 

4.1.2  Vertical  Motion  Constraint 

The  second  primary  assumption  was  that  the  relative  motion  of  the  target  and  test 
aircraft  was  constrained  to  the  vertical  axis.  This  assumption  was  made  as  a  result  of  the 
tracking  task,  applied  boundaries,  and  developed  flight  test  techniques  all  being 
constrained  to  the  aircraft  longitudinal  axis.  As  such,  the  assumption  was  that  the  motion 
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captured  on  the  aircraft  mounted  camera  would  be  primarily  vertical  in  nature.  This  was 
later  confirmed  after  reviewing  the  collected  video  footage. 

The  driver  for  this  assumption  was  the  reduction  in  the  complexity  of  the  data 
reduction  algorithm  for  the  collected  video  data.  By  limiting  the  reduction  to  a  single 
axis,  the  development  of  the  algorithm  to  determine  the  distance  to  boundary  was  greatly 
simplified.  Again,  after  reviewing  the  collected  video  footage,  this  assumption  was 
accurate  and  had  little  impact  on  the  quality  of  the  reduced  data. 

4.2  Data  Reduction  Algorithms 

Data  reduction  tools  consisted  of  both  commercial  off-the-shelf  software  and 
locally  created  MATLAB®  routines.  Each  served  a  specific  purpose  in  reducing  the 
separate  streams  of  data  and  integrating  them  into  the  required  data  products  for  analysis. 
After  each  flight,  data  reduction  generally  followed  two  tracks  in  parallel:  one  for  the 
video  data,  and  one  for  the  stick  deflection  data.  The  two  data  streams  were  integrated  at 
the  last  step  to  produce  the  final  data  products  for  analysis.  Figure  12  illustrates  the  flow 
of  data  through  the  data  reduction  process. 
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Figure  12.  Data  Reduction  Flow  Chart 


The  onboard  camera  used  to  capture  the  pilot’s  perspective  stored  the  captured 
images  on  an  analog  tape.  To  process  the  video  digitally,  a  commercial  off-the-shelf 
video  tape  player  converted  the  analog  signal  to  an  uncompressed  version  of  the  Audio 
Video  Interleave  (AVI)  file  fonnat  and  digitally  exported  the  file  to  a  computer  via  an 
Institute  of  Electrical  and  Electronics  Engineers  (IEEE)  standard  1394  bus  also  known  as 
FireWire. 

Once  the  analog  file  was  converted  to  a  digital  format,  Adobe  Premiere®  was 
used  to  reduce  the  typical  14  gigabyte  video  file  into  a  more  manageable  2  gigabyte  size. 
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This  was  done  by  deleting  the  nonessential  portions  of  the  typical  1  hour  and  15  minute 
flight.  These  portions  included  all  events  except  the  actual  test  points  which  were 
typically  10  to  15  seconds  long.  The  end  result  was  a  reduction  of  the  flights  video 
length  from  1  hour  and  15  minutes  to  roughly  7  minutes.  Two  seconds  of  video  data 
prior  to  the  beginning  of  the  maneuver  were  collected  as  a  buffer  to  determine  trim 
information  and  the  clip  was  ended  at  either  the  tennination  of  the  maneuver  or  when  the 
aircraft  departed  the  field-of-view  (FOV)  of  the  camera.  The  15  to  17  minute  AVI  clip 
contained  all  the  pertinent  events  of  the  sortie  but  was  still  in  video  format.  To  analyze 
each  individual  10  to  15  second  event,  the  AVI  clip  was  parsed  and  converted  into  a  30 
frames-per-second  (fps)  still  image  sequence  and  saved  in  individual  folders  descriptively 
labeled  to  organize  and  separate  individual  events. 

4.2.1  Automated  Visual  Tracking  Routine 

Collected  video  data  were  processed  immediately  following  each  flight.  Once 
converted  to  AVI  format,  the  video  events  were  first  processed,  using  Adobe  Premiere, 
through  three  chromatic  filters  to  help  eliminate  background  clutter  and  other  noise  items. 
A  red  filter  was  used  to  isolate  the  upper  marks,  a  yellow  for  the  lower  marks  and  a  green 
filter  was  used  to  isolate  the  wingtip.  Figure  13  illustrates  a  pre-processed  frame  (on  the 
left)  and  that  same  frame  after  the  chromatic  filtering  process  (shown  on  the  right).  The 
chromatic  filter  image  output  is  a  black-and-white  AVI  which  optimizes  the  video  for 
processing  in  Matlab" .  Each  of  these  three  files  were  processed  though  a  Matlab® 
routine  that  detennined  time  histories  of  wingtip  position,  boundary  positions,  and 
wingtip  rates  relative  to  these  boundaries  positions. 
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Figure  13.  Chromatic  Filtering  of  Visual  Data 


The  developed  Matlab'  visual  tracking  algorithm  applied  a  vanishing  point 
technique  to  detennine  the  location  of  the  boundaries  in  each  frame  of  video  data.  The 
red  and  yellow  markings  defined  two  horizontal  parallel  lines  that  on  a  two-dimensional 
frame  of  video  appear  to  intersect  at  some  distant  point,  the  vanishing  point.  Once  the 
location  of  the  vanishing  point  was  detennined,  the  location  of  the  boundaries  could  be 
determined  from  a  line  drawn  from  the  vanishing  point  to  the  boundaries  on  the  aircraft. 

With  the  boundaries  defined,  the  location  of  the  wingtip  was  determined  by 
tracking  the  pixel  position  of  the  green  chroma  filtered  image.  Unfortunately,  this  is 
where  the  automated  tracking  algorithm  broke  down.  The  wingtip  of  the  T-38  aircraft  is 
very  narrow.  Even  with  a  distinctive  green  marking,  the  aircraft  mounted  camera  system 
did  not  possess  adequate  resolution  to  identify  the  wingtip  in  each  frame  of  the  collected 
video  data.  This  prevented  the  automated  tracking  algorithm  from  tracking  the  wingtip 
position. 
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The  next  step  was  to  apply  the  automated  algorithm  to  at  least  detennine  the 
location  of  the  boundaries  and  then  manually  determine  the  location  of  the  wingtip. 
However,  the  location  of  the  vanishing  point  was  extremely  noisy  and  at  an  extreme 
distance.  This  prevented  an  accurate  detennination  of  the  location  of  the  boundaries  on 
the  target  aircraft.  Coupling  this  with  the  inability  to  accurately  determine  the  wingtip 
position  automatically,  a  manual  data  reduction  algorithm  was  required  to  extract  the 
information  from  the  visual  data. 

4.2.2  Manual  Visual  Data  Reduction  Routine 

A  manual  visual  data  reduction  tool  was  developed  by  Maj.  Scott  Heritsch,  a 
member  of  the  flight  test  team,  to  aid  in  the  extraction  of  data  from  the  collected  video 
images.  The  tool,  named  ScratchClick,  was  a  Matlab®  GUI  routine  that  allowed  the  user 
to  select  specific  points  on  each  frame  of  video  and  save  the  pixel  locations  for  further 
analysis.  To  determine  the  time-to-boundary  as  perceived  by  the  pilot,  six  points  of 
interest  per  frame  were  required:  one  at  the  leading  corner  of  the  target  aircraft  wingtip, 
one  at  the  tracking  reference  where  the  wingtip  was  to  be  maintained,  and  two  to  define 
each  linear  boundary  above  and  below  the  tracking  reference.  Figure  14  illustrates  the 
ScratchClick  interface  as  well  as  how  ScratchClick  displayed  these  six  points.  From  this 
information  the  vertical  distances  from  the  tracking  point  to  the  wingtip  and  to  each 
boundary  were  calculated. 

A  significant  amount  of  visual  data  was  collected  during  the  course  of  the  flight 
test.  As  such,  only  approximately  30%  of  the  total  data  was  reduced  and  analyzed  during 
this  research. 
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Figure  14.  Example  ScratchClick  Interface  for  Visual  Data  Extraction 


4.2.3  Aircraft  Data  Acquisition  System  Routine 

The  data  collected  by  the  aircraft  data  acquisition  system  (DAS)  was  collected  via 
an  8mm  tape.  The  data  was  then  converted  from  binary  to  numerical  data  for  analysis. 
Once  each  event  was  determined  in  the  reduction  of  the  video  data,  the  DAS  data  was 
parsed  to  match  the  duration  of  the  video  data. 
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4.3  Aircraft  Flying  Qualities  Parameter  Analysis 

4.3.1  Determination  of  System  Time  Delay 

To  obtain  the  system  time  delay  data,  the  test  aircraft  stabilized  in  the  point 
tracking  formation  position  off  of  the  target  aircraft.  The  target  aircraft  provided  a  stable 
reference  platfonn  by  maintaining  straight  and  level,  unaccelerated  flight.  When  cleared 
to  maneuver,  the  pilot  of  the  test  aircraft  performed  an  aft  stick  step  input. 

The  system  time  delay  was  calculated  as  the  length  of  time  between  two  events: 
the  pilot’s  initial  movement  of  the  stick  from  the  trim  position,  and  the  initial  response  of 
the  aircraft.  Definite  stick  movement  was  determined  by  a  rapid  stick  deflection  beyond 
the  noise  levels  observed  while  stabilized  in  the  trim  condition.  The  step  inputs  were  of 
sufficient  quality  to  readily  identify  the  moment  the  input  was  commenced.  The  system 
time  delay  was  determined  by  overlaying  the  time  history  of  the  longitudinal  stick 
position  on  the  collected  video.  The  time  at  which  the  step  input  was  made  was  recorded 
and  then  the  time  at  which  visible  aircraft  motion  first  began  was  recorded.  The 
difference  in  these  two  values  was  recorded  as  the  system  time  delay.  Table  4 
summarizes  the  system  time  delays  for  each  flight  condition.  The  time  delay  values  were 
calculated  from  one  test  run  at  each  condition. 
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Table  4.  System  Time  Delays 


Flight  Condition 

Altitude 

(feet) 

Landing  Gear 
Position 

Airspeed 

(KIAS) 

System 
Time  Delay 
(seconds) 

High  Fast  Cruise 

20,000 

Up 

375 

0.07 

High  Slow  Cruise 

20,000 

Up 

250 

0.17 

High  Configured 

20,000 

Down 

210 

0.30 

Low  Fast  Cruise 

10,000 

Up 

375 

0.03* 

Low  Slow  Cruise 

10,000 

Up 

250 

0.10 

Low  Configured 

10,000 

Down 

210 

0.17 

*  Note:  The  data  sampling  rates  limited  the  precision  of  the  system  delays  when  less  than 
1/20  of  a  second. 

4.3.2  Determination  of  System  Crossover  Frequency 

The  system  crossover  frequency,  co iso,  is  defined  for  this  experiment  as  the  rate  at 
which  an  oscillatory  pitch  input  by  the  pilot  results  in  the  opposite  vertical  movement 
from  the  aircraft.  When  cycling  the  stick  at  this  frequency,  the  aircraft  will  climb  during 
forward  stick  application  and  descend  during  aft  stick  application  because  the  aircraft  is 
still  responding  to  the  previous  input.  If  the  pilot’s  input  is  slower  than  the  crossover 
frequency,  the  aircraft  will  follow  the  stick  inputs  (aft  stick  results  in  a  climb,  as  one 
would  expect);  if  the  pilot’s  input  is  faster,  the  aircraft  does  not  have  enough  time  to 
respond  to  the  rapidly  alternating  commands  and  remains  nearly  level  at  the  trim  state. 
The  system  crossover  frequency  is  of  interest  because  if  the  pilot  makes  repeated 
corrections  at  this  rate  (such  as  when  bouncing  between  two  opposing  boundaries),  the 
aircraft  may  move  in  a  direction  opposite  what  the  pilot  is  commanding  and  anticipating. 
Such  a  situation  increases  the  potential  for  divergent  oscillations  and  PIOs. 
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To  determine  the  system  crossover  frequency,  the  test  aircraft  performed  a 
frequency  sweep  in  pitch  while  at  or  near  the  standard  formation  position  off  the  target 
aircraft.  To  perform  the  frequency  sweep,  the  pilot  made  slow,  alternating  inputs  of 
small  stick  deflections  and  gradually  increased  the  frequency.  Once  the  pilot  perceived 
the  aircraft  vertical  movement  to  be  out  of  phase  with  the  stick  deflection,  the  pilot 
dwelled  at  that  frequency  to  record  a  sufficient  stick  trace  on  the  DAS  for  later  analysis. 
Throughout  the  frequency  sweep,  the  pilot  adjusted  stick  deflections  to  keep  the  target 
aircraft  within  the  field  of  view  of  the  aircraft-mounted  camera  for  post-flight  video 
analysis.  Post- flight  analysis  entailed  measuring  the  period  of  the  longitudinal  stick 
oscillations  and  taking  the  reciprocal  to  determine  com-  The  measured  system  crossover 
frequencies  are  listed  in  table  5. 


Table  5.  System  Crossover  Frequencies 


Flight  Condition 

Altitude 

(feet) 

Landing 

Gear 

Position 

Airspeed 

(KIAS) 

System 

Crossover 

Frequency 

(Hz) 

High  Fast  Cruise 

20,000 

Up 

375 

0.88 

High  Slow  Cruise 

20,000 

Up 

250 

0.59 

High  Configured 

20,000 

Down 

210 

0.56 

Low  Fast  Cruise 

10,000 

Up 

375 

0.71 

Low  Slow  Cruise 

10,000 

Up 

250 

0.45 

Low  Configured 

10,000 

Down 

210 

0.45 
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4.4  Boundary  Avoidance  Tracking  Characterization 
4.4.1  BAT  Data  Analysis  Tool 

The  reduced  data  from  both  the  aircraft  mounted  camera  system  and  the  aircraft 

(R) 

data  acquisition  system  had  to  be  collated  to  fully  analyze  each  BAT  event.  A  Matlab 
graphical  user  interface  (GUI)  tool,  BATCalc,  was  created  to  pull  both  the  visual  data 
and  the  DAS  data  together  and  provide  a  single  interface  to  determine  all  of  the 
parameters  of  interest.  Figure  15  shows  the  BAT  Calc  tool  that  was  developed.  There 
are  several  displays  and  controls  on  this  tool,  and  each  will  be  discussed  more  fully  in  the 
sections  that  follow. 
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Figure  15.  BAT  Data  Analysis  Tool  -  BATCalc 


BATCalc  provided  four  primary  functions:  estimation  of  longitudinal  stick  trim 
position,  estimation  of  pilot  actuation  frequency,  calculation  of  distance  to  boundary  from 
visual  data,  and  collation  of  visual  and  DAS  data.  This  section  will  focus  on  the 
calculation  of  distance  to  boundary  from  visual  data  and  the  collation  of  visual  and  DAS 
data.  Estimation  of  longitudinal  stick  trim  position  and  determination  of  pilot  actuation 
frequency  will  be  discussed  in  subsequent  sections. 
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The  visual  data  provided  by  the  ScratchClick  tool  provided  the  pixel  positions  of 
the  boundaries,  the  wingtip,  and  the  center  of  the  chevron  star.  Two  points  each  were 
used  to  define  the  upper  and  lower  boundaries,  one  point  for  the  wingtip,  and  one  point 
for  the  center  of  the  chevron  star.  Figure  16  details  how  the  points  were  selected  in 
ScratchClick  and  how  they  are  referenced  in  the  BAT  Calc  GUI. 


Figure  16.  Schematic  of  Visual  Data  Point  Selection 

Points  1  and  2  were  used  to  determine  the  equation  of  the  line  that  defined  the 
upper  boundary;  likewise,  points  5  and  6  were  used  for  the  lower  boundary.  Point  3,  the 
center  of  the  chevron  star,  was  used  as  the  center  point  of  the  tracking  task.  The  x- 
coordinate  of  point  3  was  used  to  draw  a  vertical  line  that  intersected  both  the  upper  and 
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lower  boundaries.  Those  points  of  intersection  then  defined  the  pixel  distance  of  each 
boundary  from  the  center  of  the  tracking  task. 

The  wingtip,  point  4,  was  assumed  to  track  along  the  vertical  line  calculated  from 
point  3  throughout  the  event.  The  distance  from  the  tracking  task  center  point  was  simply 
the  difference  in  y-coordinate  values  between  points  3  and  4.  The  following  equations 
detail  the  algorithm  described  above.  The  subscripts  of  each  x  and  y  variable  reference 
the  points  as  shown  on  Figure  16. 
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Where: 

m  =  the  slope  of  the  specified  line 
y  =  vertical  pixel  position  of  specified  point 
x  =  horizontal  pixel  position  of  the  specified  point 
d  =  pixel  distance  between  two  pixel  positions 

Note:  pixel  positions  were  numbered  from  the  top  left  comer  of  each  image; 
therefore,  to  produce  boundaries  about  the  zero  reference  point,  y3,  the  d/ower  value 
was  calculated  as  a  negative  distance 


This  algorithm  was  then  applied  to  each  frame  of  visual  data  to  develop  a  time 
history  of  each  event.  An  example  time  history  is  provided  in  Figure  17,  where  the 
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distance  to  boundary  is  highlighted.  The  solid  line  in  the  middle  of  the  plot  shows  the 
position  of  the  wingtip;  the  dashed  line  on  the  top  of  the  plot  shows  the  lower  boundary; 
and  the  solid  line  at  the  bottom  of  the  plot  shows  the  upper  boundary.  The  position  of  the 
boundaries  was  reversed  to  better  reflect  the  relative  position  of  the  test  aircraft.  If  the 
wingtip  is  above  zero,  then  the  test  aircraft  is  above  the  target  aircraft  and  the  target 
aircraft  wingtip  appears  to  be  closer  to  the  lower  boundary,  and  vice  versa.  This 
presentation  was  more  intuitive  to  the  flight  crew  during  the  post  flight  data  analysis. 
With  the  position  infonnation  now  determined,  the  time-to-boundary  can  be  calculated. 
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Figure  17.  Distance  to  Boundary  Time  History  -  BAT  Calc  Display 


As  show  in  equation  2,  the  time-to-boundary  was  calculated  from  both  the 
distance  to  boundary  and  the  rate  to  boundary.  The  distance  to  boundary  was  calculated 
directly  from  the  position  infonnation  obtained  from  the  position  time  history. 
Consequently,  the  distance  to  boundary,  Dboundary,  for  both  the  upper  and  lower 
boundaries  was  simply  determined  from  the  difference  in  the  y-coordinates  of  the  wingtip 
and  the  boundaries. 
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D  upper  d upper  ^ wingtip  T  4  y ub 
D lower  —  d lower  ^ wingtip  T  4  y lb 


(5) 


Where: 

D Upper  =  pixel  distance  of  target  wingtip  to  upper  boundary 
D tower =  pixel  distance  of  target  wingtip  to  lower  boundary 


The  rate  to  each  boundary  was  detennined  through  the  numerical  derivative  of  the 
distance  to  boundary.  Hence,  the  derivative  was  approximated  by: 


AD 


boundary 


' boundary 


At 
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Where: 

f i boundary  rate  to  boundary 


Once  both  the  distance  and  rate  to  boundary  values  were  calculated,  the  time-to- 
boundary  was  calculated  by  dividing  the  distance  to  boundary  by  the  rate  to  boundary. 

An  important  note  here  is  that  the  distance  and  rate  to  boundary  values  were  calculated 
based  on  pixel  distances;  that  is,  the  difference  between  pixel  locations  on  one  frame  of 
the  visual  data.  Since  the  assumption  was  made  that  the  visual  data  provided  by  the 
aircraft  mounted  camera  system  was  representative  of  what  the  pilot  viewed  during  the 
BAT  events,  the  pixel  distances  were  assumed  to  reflect  the  actual  visual  distances  the 
pilots  were  tracking.  Therefore,  the  actual  physical  distances  were  not  required  for  the 
determination  of  the  time-to-boundary.  Figure  18  shows  a  representative  plot  of  the  time- 
to-boundary  as  displayed  in  the  BAT  Calc  GUI.  The  dashed  line  references  the  time  to 
the  lower  boundary  and  the  solid  line  the  time  to  the  upper  boundary. 
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Figure  18.  Time  to  Boundary  Time  History  -  BATCalc  Display 

The  only  remaining  parameter  required  to  detennine  all  of  the  BAT  parameters  of 
interest  is  the  longitudinal  stick  position. 

4.4.2  Estimating  Longitudinal  Stick  Trim  Position 

Pilot  system  inputs  were  measured  via  the  longitudinal  stick  displacement 
recorded  by  the  aircraft  DAS.  The  stick  displacement  was  measured  at  a  rate  of  20 
samples  per  second  and  had  a  range  of  approximately  45  degrees.  On  the  ground  the 
stick  was  centered  at  approximately  5  degrees  forward  stick;  however,  in  flight  the  stick 
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was  centered  about  a  trim  position  for  each  flight  condition.  The  developed  fight  test 
techniques  also  required  a  rapid  change  in  flight  condition  leading  to  time-dependent  trim 
positions.  Thus,  a  method  for  estimating  the  stick  trim  position  resulting  from  rapidly 
changing  flight  conditions  was  required  to  more  accurately  present  the  magnitudes  of  the 
pilot  inputs. 

The  BATCalc  tool  utilized  a  moving  point  averaging  technique  developed  at  the 
National  Aeronautics  and  Space  Administration’s  Dry  den  Flight  Research  Center  to 
estimate  the  longitudinal  stick  trim  position.  This  Matlab®  function,  titled  MAVE, 
averaged  the  specified  number  of  points  before  and  after  the  point  of  interest  and  repeated 
this  for  each  point  in  the  specified  array.  MAVE  allowed  for  the  input  of  the  number  of 
points  to  average  around  each  data  point.  In  the  BAT  Calc  GUI,  this  MAVE  value  was 
adjusted  until  a  suitable  fit  to  the  data  was  obtained.  Figure  19  depicts  the  plots  of  both 
the  collected  data  parameters  (solid  lines)  and  the  averaged  trim  histories  (dashed  lines). 
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Figure  19.  Longitudinal  Stick  Trim  Position  -  BAT  Calc  Display 


To  aid  in  establishing  a  suitable  fit,  the  aircraft  airspeed  and  pitch  attitude  were 
used  along  with  the  stick  position  (see  Figure  19).  In  fact,  the  pitch  attitude  and  the 
airspeed  were  used  primarily  to  establish  the  value  of  the  MAVE  function  as  those  two 
parameters  were  used  by  the  target  aircraft  pilots  to  execute  the  flight  test  techniques. 
Once  a  satisfactory  fit  was  obtained,  the  longitudinal  stick  trim  position  time  history  was 
established.  From  this  point,  the  longitudinal  stick  position  used  for  the  determination  of 
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the  BAT  parameters  was  the  difference  between  the  DAS  recorded  stick  position  and  the 
estimated  trim  stick  position. 

4.4.3  Identification  of  Kmax  and  tKmax 

With  the  stick  trim  position  estimated  and  the  time  histories  developed,  the 
identification  and  detennination  of  the  key  BAT  parameters  can  be  conducted.  The 
easiest  BAT  parameters  to  identify  in  the  collected  data  were  the  maximum  boundary 
tracking  feedback  gain,  Kmax,  and  the  time-to-boundary  for  the  maximum  boundary 
tracking  feedback  gain,  tKmax-  Kmax  was  simply  recorded  as  the  stick  deflection  from  trim 
at  the  point  when  a  maximum  stick  deflection  occurred  during  a  BAT  event.  The  tKmax 
parameter  was  then  the  time  to  boundary  that  corresponded  to  the  maximum  stick 
deflection.  Figure  20  shows  how  these  two  values  were  taken  from  the  BAT  Calc  GUI. 
Once  the  maximum  longitudinal  stick  position  was  identified  from  the  stick  position  time 
history,  a  vertical  line  was  drawn  and  the  time-to-boundary  associated  with  that  stick 
position  was  recorded  as  tKmax-  If  the  stick  position  data  point  was  between  two  time-to- 
boundary  data  points,  then  linear  interpolation  was  used  to  determine  the  time-to- 
boundary. 


61 


a  <Student  Version>  :  BAT_Calc  [~-~f  □ 

r  BAT  Event — 


Figure  20.  Determination  of  Kmax  and  txmax  -  Bat  Calc  Display 

It’s  important  to  note  that  the  value  of  Kmax,  as  defined  in  equation  2,  corresponds 
to  a  gain  value  used  to  model  the  BAT  feedback  loop.  The  Kmax  determined  from  the 
flight  test  data  is  not  a  direct  analogy  to  the  Kmax  originally  defined  by  Mr.  Gray.  To 
determine  the  analogous  value  of  Kmax,  the  BAT  model  should  be  fitted  to  the  flight  test 
data.  That  was  not  accomplished  during  this  experiment  and  is  an  area  that  can  be 
investigated  in  the  future.  Nonetheless,  the  values  of  the  maximum  stick  deflections  and 
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the  associated  times-to-boundary  were  determined  as  shown  in  Figure  20  and  are 
summarized  in  the  Appendix. 

4.4.4  Determination  of  tKmin 

The  time-to-boundary  for  minimum  boundary  tracking  feedback  gain,  or  tKmin, 
proved  to  be  the  most  difficult  parameter  to  identity.  Originally  it  was  theorized  that  a 
definitive  event  in  stick  position  (such  as  a  flattening  or  rapid  reversal)  would  provide  the 
cue  for  the  determination  of  the  tKmin •  However,  that  distinctive  event  was  never  apparent 
in  any  data  set.  A  method  had  to  be  developed  that  would  allow  for  the  determination  of 
this  key  parameter. 

Methods  involving  stick  velocity,  stick  acceleration,  load  factor,  and  rate  of 
change  of  load  factor  were  investigated  as  potential  cues  for  tKmin ■  The  acceleration  of  the 
longitudinal  stick  position  proved  to  be  repeatable  and  reliable  at  providing  a  time-to- 
boundary  at  which  a  BAT  event  began.  This  method  correlated  a  large  stick  acceleration 
in  the  direction  of  the  boundary  with  a  time-to-boundary  greater  than  tKmax- 

The  stick  acceleration  is  a  direct  indicator  of  the  pilot’s  applied  force.  Since 
pilots  apply  forces  to  control  inceptors  in  order  to  obtain  desired  inceptor  positions  and 
therefore  desired  aircraft  responses,  stick  force  is  a  good  indication  of  pilot  intent. 
Therefore,  looking  at  the  point  where  the  pilot’s  applied  stick  force  made  a  sharp  increase 
opposite  the  approaching  boundary  was  a  good  indication  of  the  point  at  which  the  pilot 
first  attempted  to  avoid  the  boundary. 

The  stick  acceleration  was  calculated  by  differentiating  the  stick  position  data 
twice.  It  was  then  plotted  in  the  BAT  Calc  GUI  for  detennination  of  tKmin ■  Figure  21 


63 


depicts  the  same  BAT  event  as  that  pictured  in  Figure  20.  However,  the  stick  position 
has  been  replaced  with  the  stick  acceleration.  Here  the  pilot  applies  an  aft  stick  force  as 
the  aircraft  begins  to  descend  relative  to  the  target  aircraft  in  an  effort  to  arrest  the  closure 
to  the  impending  boundary. 


Figure  21.  Determination  of  tKmin  -  BATCalc  Display 


Two  important  notes  resulted  from  the  determination  of  tKmin ■  The  first  was  that 
there  appeared  to  be  a  learning  curve  during  the  BAT  events.  With  events  that  had 
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multiple  boundary  avoidance  occurrences,  the  values  associated  with  each 
occurrence  generally  decreased  in  value  as  the  events  progressed.  This  is  potentially 
important  as  tKmm  may  be  defined  by  the  very  first  occurrence  of  boundary  avoidance 
tracking  and  not  independently  at  each  occurrence  during  a  BAT  event.  If  this  is  true, 
much  more  data  will  have  to  be  analyzed  to  characterize  Am,-„  and  its  relationship  with 
other  parameters.  The  values  of  determined  and  presented  in  Appendix  A  are  the 
values  associated  with  each  occurrence  of  each  BAT  event. 

The  second  note  is  that  in  some  of  the  data  the  times-to-boundary  appeared  to  be 
affected  by  the  aircraft  load  factor.  To  evaluate  this  relationship,  the  load  factor  at  the 
point  of  boundary  awareness  was  recorded  during  the  data  reduction  as  well.  The 
analysis  of  this  relationship  will  be  presented  in  section  4.4.6. 

4.4.5  Pilot  Actuation  Frequency  Determination 

To  investigate  the  proximity  of  the  system  operating  frequency  to  the  system 
crossover  frequency,  determination  of  the  frequency  at  which  the  pilot  was  providing 
inputs  to  the  system  was  required.  To  calculate  this  parameter,  a  power  spectral  density 
(PSD)  was  performed  on  the  longitudinal  stick  position  data.  The  detennined  pilot 
actuation  frequency  was  assumed  constant  for  the  duration  of  the  BAT  event; 
consequently,  regardless  of  the  number  of  occurrences  within  a  single  event,  the  pilot 
actuation  frequency  was  the  same. 

The  applied  technique  for  the  determination  of  the  PSD  was  a  Matlab  ®  function 
again  obtained  from  NASA  DFRC.  The  function  utilized  the  Matlab1'  built-in  fast 
Fourier  transform  function,  FFT,  to  calculate  the  frequency  domain  information  of  the 
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longitudinal  stick  position.  Once  the  data  were  converted  to  the  frequency  domain,  the 
function  then  computed  the  amplitude  by  first  multiplying  the  FFT  output  amplitude  by 
its  complex  conjugate  and  dividing  the  result  by  the  number  of  data  points.  Then  the  data 
was  smoothed  by  the  MAVE  function  using  a  5  point  average.  Finally,  since  the  Fourier 
transfonn  calculated  the  amplitude  data  as  symmetric  about  zero  frequency,  half  of  the 
data  was  then  multiplied  by  the  time  differential  to  compute  the  PSD  of  the  longitudinal 
stick  position  time  history.  The  resulting  plot  from  this  algorithm  was  then  displayed  on 
the  BAT  Calc  GUI  allowing  the  user  to  determine  the  pilot  actuation  frequency  (see 
Figure  22).  The  pilot  actuation  frequencies  for  each  analyzed  event  are  located  in  the 
Appendix. 
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Figure  22.  PSD  of  Longitudinal  Stick  Position  -  BATCalc  Display 


4.4.6  Analysis  of  BAT  Parameters 

Analysis  of  the  BAT  parameters  of  interest,  tKmin,  fomax,  and  Kmax,  started  with 
statistical  analyses  of  each  parameter.  Parameter  mean,  variance,  and  95%  confidence 
interval  on  the  mean  were  considered  during  the  analyses. 

4.4.6. 1  Time  to  Boundary  for  Minimum  Boundary  Feedback  Gain  (tKmin) 
The  first  parameter  analyzed  was  tKmin-  First,  all  the  data  collected  from  the 
BAT_Calc  tool  were  analyzed  to  detennine  if  tKmin  possessed  a  central  tendency  and  to 
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compare  that  value  to  the  results  of  Mr.  Gray’s  simulator  study.  Statistical  analysis  for  a 
total  of  36  data  points  showed  that  no  central  tendency  existed.  Furthermore,  the  95% 
confidence  interval  on  the  mean  was  only  0.78,  or  22%  confident  that  the  calculated 
mean  was  representative  of  the  data  set. 

Each  boundary  avoidance  occurrence  required  the  aircrew  to  either  increase  load 
factor  (pull)  or  decrease  load  factor  (push)  to  avoid  the  impending  boundary.  The  values 
collected  for  tKmm  were  also  divided  into  “push”  and  “pull”  categories  to  detennine  if  tKmin 
exhibited  a  central  tendency  when  either  the  load  factor  was  increasing  or  decreasing. 

The  results  again  indicated  no  central  tendency  in  either  push  or  pull.  Table  6  presents 
the  results  of  the  statistical  analyses. 


Table  6.  Results  of  Statistical  Analyses  for  tKmin 


Mean 

Standard  Deviation 
Sample  Variance 
Count 

95%  Confidence 
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If 

2.54 
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6.48 

36 

13 

23 

0.78 

0.55 

ai 

4.17 


The  inconclusive  results  of  the  statistical  analyses  of  tKmin  required  looking  at 
possible  relationships  between  this  and  other  cueing  parameters.  Based  on  the 
observations  made  during  the  determination  of  tKmin  (section  4.4.4)  that  time-to-boundary 
seemed  to  vary  with  load  factor,  this  was  the  relationship  that  was  investigated.  The  load 
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factor  and  rate  of  change  of  the  load  factor  at  each  tKmin  point  was  collected  utilizing  the 
BATCalc  GUI. 

The  nuisance  parameter  study  conducted  in  2001  provided  a  time  to  boundary  of 
approximately  two  seconds  for  pilots  to  begin  feeling  nervous  about  the  approach  to  the 
ground  (Prosser,  2001 : 1 1).  Interestingly,  that  study  seemed  to  indicate  a  somewhat 
constant  time  to  boundary  for  the  specific  scenario  they  established.  That  number 
correlates  well  with  Mr.  Gray’s  simulator  results  of  approximately  2.8  seconds  (Gray, 
2005:1 1).  However,  neither  study  investigated  the  effect  of  load  factor  on  the  pilot’s 
perception  of  time  to  boundary. 

4.4.6.2  Correlating  tKmm  and  Load  Factor 
To  determine  if  a  relationship  existed,  tKmin  was  plotted  against  both  the  load 
factor  at  tKmin,  GKmin,  the  rate  of  change  of  load  factor  at  tKmin,  (AG/ At) Kmin,  as  well  as 
variations  of  the  products  and  quotients  of  these  parameters.  First,  simply  plotting  tKmin 
and  GKmin  against  each  yielded  the  results  shown  in  the  Figure  23. 
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Figure  23.  Correlation  of  tKmin  and  Gamin 

The  plot  shows  a  wide  scatter  in  the  values  of  tKmin  with  respect  to  the  load  factor 
at  which  tKmin  occurred.  A  linear  regression  was  perfonned  (line  B)  and  provided  a  poor 
R-squared  value  indicating  a  poor  fit  of  the  data.  However,  this  linear  model  compared 
somewhat  favorably  with  results  of  Mr.  Gray’s  simulator  study.  Mr.  Gray’s  fixed-based 
simulator  provided  average  results  of  2.8  ±  0.25  seconds  for  tKmin  (Gray,  2005: 1 1).  Using 
the  above  equation  for  the  linear  model  and  applying  a  load  factor  of  one  resulted  in  a 
tKmin  value  of  3.54  seconds.  With  only  a  20%  difference  between  the  two  values,  Mr. 
Gray’s  results  seem  to  be  relatively  close  to  what  was  seen  during  the  flight  tests. 
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During  the  investigation  of  the  many  Ga„  and  tumm  combinations,  only  one  other 
relationship  showed  any  degree  of  correlation.  This  relationship,  shown  in  Figure  24, 
was  developed  by  normalizing  both  Gamin  and  txmm  by  the  rate  of  change  of  the  load 
factor,  ( AG/At)Kmin ■  (AG/At)Kmm  was  selected  as  a  parameter  of  interest  for  characterizing 
BAT  events  because  pilots  typically  respond  not  just  to  load  factor,  but  also  to  how 
quickly  the  load  factor  is  changing.  A  linear  regression  of  the  data  provided  for  a  much 
better  curve  fit  than  that  obtained  with  the  previous  relationship. 
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Constraining  the  linear  regression  to  a  zero  y-intercept  provided  for  a  minor 
difference  in  the  quality  of  the  curve  fit,  as  shown  in  the  Figure  24  (lines  A  and  B). 
Therefore,  regression  B  was  used  for  comparison  as  it  simplified  the  relationship  to  a 
constant  without  greatly  affecting  the  relationship  to  the  collected  data.  The  result 
provided  for  the  following  relationship: 

t  .  G  .  t  .  IG  .  t  . 

min  _  g  *  mm  mm  /  mm  _  g  min  _  g  /  n  > 

( AG\  ( AG\  ( AG^\  /  ( AG']  G  .  K 

-  /  -  min 

v  At  )  y  At  J  y  At  J I  y  At  y 

Where: 

B  =  5.18  (constant  determined  from  the  linear  regression) 

Applying  this  simplified  relationship  to  the  case  of  a  load  factor  of  one  resulted  in 
a  tKmin  value  of  5.18  seconds.  Unlike  the  previous  linear  regression  results,  this  result  did 
not  correlate  well  with  the  average  tKmin  value  obtained  from  Mr.  Gray’s  simulator  study. 

Going  back  to  the  original  investigation  comparing  tKmin  and  directly  and 
applying  the  same  zero  y-intercept  linear  regression  resulted  in  an  even  poorer  curve  fit 
(Figure  23,  line  A).  However,  for  a  load  factor  of  one,  the  tKmin  parameter  was  2.57 
seconds  which  again  compared  favorably  with  the  results  of  Mr.  Gray’s  simulator  study. 
While  the  direct  comparison  results  seem  to  correlate  relatively  well  with  Mr.  Gray’s 
simulator  study  results,  they  do  not  correlate  well  with  the  relationship  developed  using 
the  normalized  parameters,  which  appears  to  be  more  representative  of  the  collected  data. 
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To  conclude  this  investigation,  statistical  analyses  of  the  ratio  of  Gfcmm  to  tKmin 
were  conducted  to  determine  if  a  central  tendency  existed  in  the  relationship.  If  a  strong 
central  tendency  existed,  then  the  theory  of  a  constant  that  describes  the  relationship 
between  tKmin  and  load  factor  would  be  strengthened.  Performing  the  same  analyses  as 
for  the  original  tKmin  investigation  provided  the  results  shown  in  table  7. 


Table  7.  Statistical  Analyses  of  the  Ratio  of  GKmil,  to  tKmin 


G  Kmin! tKmin  1  GfCmin  /  tKmin  1 

(G Kmin  /  t Kmin )  /  1 

(AG/ At)  Kmin 

Mean 

0.30 

0.35 

0.47 

Standard  Deviation 

0.40 

0.36 

0.40 

Sample  Variance 

0.16 

0.13 

0.16 

Count 

36 

36 

36 

95%  Confidence 

0.13 

0.12 

0.14 

From  these  analyses  it  is  clear  that  there  is  a  relationship  between  the  time  to 
boundary  for  minimum  boundary  tracking  feedback  gain  and  the  load  factor  at  the  time  of 
boundary  awareness.  The  exact  nature  of  the  relationship  stills  bears  some  investigation, 
but  as  a  first  order  approximation  to  this  relationship,  the  author  suggests: 


=  5.18 


(8) 


This  relationship  provides  for  the  best  representation  of  the  collected  data  and  best 
represents  what  was  experienced  while  in  flight. 
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4.4.6.3  Time  to  Boundary  for  Maximum  Boundary  Feedback  Gain  (tKmax) 

The  next  parameter  analyzed  was  tKmax •  The  same  statistical  analyses  were 
performed  on  the  collected  values  of  tKmax  as  those  perfonned  on  tKmm,  however,  very 
different  results  were  obtained.  The  analysis  indicated  that  tKmax  was  an  independent 
parameter.  The  data  exhibited  a  fairly  strong  central  tendency  with  a  mean  value  of  0.78 
±  0. 18  seconds.  The  95%  confidence  interval  was  0. 14  or  86%  confident  that  the  mean 
was  representative  of  the  data  set. 

Again  similar  to  the  analysis  for  tKmm,  the  tKmax  data  points  were  divided  into  pull 
and  push  categories  to  detennine  if  there  was  a  variation  in  tKmax  based  on  increasing  or 
decreasing  load  factor.  As  shown  in  the  table  8,  the  analysis  shows  a  small  difference 
between  the  values.  This  difference  is  within  the  variance  of  the  mean  for  all  of  the 
collected  data;  furthennore,  when  the  total  number  of  data  points  is  split  into  two  separate 
categories,  the  number  of  data  points  in  each  category  is  too  small  to  provide  a 
statistically  significant  solution.  Therefore,  based  on  the  data  available,  there  is  no 
appreciable  difference  in  time  to  boundary  for  maximum  boundary  tracking  feedback 
gain  (tKmax)  between  increasing  or  decreasing  load  factors.  In  summary,  it  appears  that 
tKmax  is  a  constant  value  for  all  pilots  at  all  flight  conditions  evaluated  in  this  experiment. 
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Table  8.  Statistical  Analyses  of  tsmax 


Mean 

Standard  Deviation 
Sample  Variance 
Count 

95%  Confidence 


lllsl 


0.78 


0.82 


0.42 

0.44 

0.41 

0.18 

0.20 

0.17 

34 

13 

21 

1  0.14 

0.27 

0.19 

HI 

0.75 


4.4.6.4  Maximum  Boundary  Feedback  Gain  (Kmax) 

The  final  BAT  parameter  to  be  detennined  was  the  maximum  boundary  tracking 
feedback  gain.  As  stated  previously,  this  gain  was  assumed  to  be  proportional  to  the 
longitudinal  stick  position.  The  results  of  the  analysis  of  the  longitudinal  stick  position 
were  assumed  to  provide  a  representative  solution  of  the  actual  Kmax  parameter. 

Applying  the  same  type  of  statistical  analyses  to  the  longitudinal  stick  position 
data  as  applied  to  both  tKmm  and  (Kmax  yielded  inconclusive  results.  The  stick  positions  at 
both  tKmin  and  txmax  were  evaluated  as  well  as  the  difference  between  the  two  parameters. 
The  parameters  were  also  divided  up  into  push  and  pull  categories  with  similarly 
inconclusive  results.  Table  9  lists  the  results  of  the  statistical  analyses. 


Table  9.  Statistical  Analyses  of  Boundary  Feedback  Gain 


Kmin 

Kmin 

Kmin 

Kmax 

Kmax 

Kmax 

AK 

AK 

AK 

(pull) 

(push) 

(pull) 

(push) 

(pull) 

(push) 

Mean 

0.24 

0.01 

0.37 

0.66 

-4.45 

3.83 

0.32 

3.28 

-4.46 

Standard  Deviation 

2.33 

2.15 

2.46 

4.72 

2.54 

2.34 

4.60 

2.35 

2.99 

Sample  Variance 

5.43 

4.62 

6.08 

22.31 

6.44 

5.47 

21.15 

5.51 

8.97 

Count 

36 

13 

23 

34 

13 

21 

34 

21 

13 

95%  Confidence 

0.76 

1.17 

1.01 

1.59 

1.38 

1.00 

1.55 

1.00 

1.63 
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Unfortunately  this  parameter  is  not  as  straightforward  as  a  pilot’s  perceived  time 
to  boundary.  The  magnitude  of  the  longitudinal  stick  deflection  is  a  function  of  the 
pilot’s  threat  perception  and  comfort  level.  As  stated  in  Chapter  3,  the  pilots  in  this 
experiment  attempted  to  role  play  the  tracking  task  by  treating  the  boundaries  as 
hazardous.  This  was  an  attempt  to  allow  the  pilots  to  use  the  maximum  longitudinal  stick 
deflection  possible.  In  contrast,  in  all  of  the  data  analyzed,  no  pilot  ever  reached  the 
maximum  allowable  stick  deflection  during  any  tracking  task.  This  indicates  that  the 
pilots  were  self-limiting  the  magnitude  of  their  stick  inputs  to  prevent  an  uncomfortable 
situation.  This  self-limiting  confounded  the  results  of  the  analyses  of  the  stick  deflection 
with  each  pilot. 

4 A. 6.5  Time  Between  tKmin  and  tsmax 

In  addition  to  all  of  the  collected  parameters,  the  time  from  the  first  BAT  input  to 
the  maximum  BAT  input,  At,  was  recorded  for  each  BAT  occurrence.  This  parameter 
was  intended  to  be  used  in  the  development  of  characteristic  and  non-dimensional  BAT 
parameters;  however,  when  a  statistical  analysis  was  performed  on  this  parameter,  a 
surprising  result  was  obtained.  It  was  found  that  the  data  exhibited  a  strong  central 
tendency,  indicating  that  this  time  differential  was  fairly  consistent  among  all  pilots  and 
flight  conditions. 

The  value  of  this  parameter  was  determined  to  be  0.55  ±0.13  seconds.  With  the 
typical  variability  in  the  time  delay  inherent  in  each  pilot  and  the  difference  in  aircraft 
time  delay  with  each  flight  condition,  the  fact  that  this  parameter  exhibited  some 
independence  was  indeed  surprising.  This  result  potentially  points  to  another  intrinsic 
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human  characteristic;  that  once  an  individual  recognizes  a  hazardous  situation,  they  will 
instinctively  use  their  maximum  effort  to  avoid  the  situation  within  0.55  ±  0.13  seconds. 

4.5  Correlation  of  Flying  Qualities  and  Boundary  Avoidance  Tracking 

After  determination  of  the  BAT  and  flying  qualities  parameters,  the  final  step  was 
to  investigate  the  relationship  between  the  two  sets  of  parameters.  All  of  the  BAT 
parameters  were  plotted  against  each  of  the  flying  qualities  parameters  to  determine  if 
any  correlation  existed.  To  limit  the  dispersion  of  the  collected  data  about  each  flight 
condition,  the  average  values  of  each  parameter  at  each  flight  condition  were  used  for  this 
analysis. 

Only  one  potential  correlation  between  the  BAT  parameters  and  aircraft  flying 
qualities  was  evident  from  the  collected  data.  Figure  25  shows  the  relationship  between 
GKmJtKmin  and  the  system  crossover  frequency,  a>i so¬ 
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Figure  25.  Correlation  of  GKmJtKmin  with  System  Crossover  Frequency 

While  this  relationship  presents  a  possible  correlation  between  BAT  and  flying 
qualities,  further  analysis  is  required  before  this  relationship  can  be  solidified.  Since  the 
BAT  parameters  were  averaged  for  each  flight  condition,  the  number  of  data  points  at 
each  flight  condition  was  very  small,  between  only  two  and  ten.  With  such  small  sample 
populations,  the  average  values  are  very  likely  not  representative.  Also,  the  applied 
technique  for  the  determination  of  the  system  crossover  frequency  did  not  provide  a  high 
level  of  confidence  in  the  determined  values. 

Most  importantly,  what  this  correlation  shows  is  that  the  factors  that  characterize 
pilot  boundary  awareness  in  the  dynamic  flight  environment  are  extremely  complex. 
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While  Ik  max  appears  to  be  an  intrinsic  human  characteristic,  Ik  min  is  at  minimum  a  function 
of  both  load  factor  and  a  characteristic  system  frequency. 

In  addition  to  the  graphical  correlation  just  discussed,  one  other  relationship  was 
determined  through  statistical  analyses.  The  ratio  of  the  pilot  actuation  frequency,  cobat, 
to  the  system  crossover  frequency,  (Om,  was  found  to  have  a  very  strong  central 
tendency.  The  ratio  was  found  to  be  1 . 12  ±  0.06  with  a  95%  confidence  level  of  0.08. 
This  result  seems  to  indicate  that  the  pilots  operated  very  close  to  the  system  crossover 
frequency. 

However,  the  size  of  the  data  population  for  this  analysis  was  small  and  the  same 
low  confidence  crossover  frequency  data  were  used  in  the  analysis.  As  referenced 
previously,  the  pilot  actuation  frequency  was  assumed  to  be  constant  through  each  BAT 
event.  A  total  of  12  events  and  36  occurrences  were  evaluated  and  the  determined  pilot 
actuation  frequencies  were  assumed  constant  throughout  each  event;  so,  only  12  different 
values  for  cobat  were  calculated  and  compared  with  six  values  of  (Dm-  Therefore,  further 
data  reduction  is  required  to  make  a  more  conclusive  statement  about  this  relationship. 
Preliminarily  though,  this  result  may  help  explain  the  divergent  oscillations  experienced 
by  the  aircrews. 

4.6  Aircrew  Comments 

4.6.1  Flying  Qualities  Effects 

During  execution  of  the  flight  testing,  all  pilots  noted  that  a  particular  flight 
condition  appeared  to  produce  more  oscillatory  BAT  events  than  the  others.  Though  this 
flight  condition  was  different  for  each  pilot,  these  initial  observations  seemed  to  provide 
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some  support  for  the  theory  of  flying  qualities  dependence  of  the  BAT  parameters. 
Unfortunately  the  data  analyzed  here  does  not  support  this  assumption.  The  data  were, 
however,  extremely  limited  in  the  aircraft  flying  qualities  parameters  with  only  two 
parameters  evaluated.  It  is  possible  that  a  correlation  does  exist,  but  further  analyses 
including  several  other  aircraft  flying  quality  parameters  needs  to  be  accomplished. 

4.6.2  PIOs  Resulting  From  Repeated  Boundary  Avoidance  Tracking  Events 

All  pilots  described  the  rapid  oscillatory  motion  of  repeated  BAT  occurrences  as 
PIOs.  While  attempting  to  avoid  the  opposing  boundaries,  the  pilots  experienced  a  sense 
of  uncontrollability  of  the  aircraft.  They  applied  the  PIO  recovery  techniques  of  freezing 
or  releasing  the  controls  once  a  boundary  excursion  occurred.  The  boundaries  obviously 
acted  to  increase  the  pilots’  gains  during  the  tracking  tasks  and  potentially  drove  them 
close  to  the  system  crossover  frequency  rendering  the  pilot’ s-eye  system  unstable. 
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5.  Discussion 


5.1  Summary  of  Findings 

Overall,  this  experiment  showed  that  the  presence  of  boundaries  can  provide  a 
destabilizing  influence  and  hinder  a  pilot’s  ability  to  successfully  accomplish  a  specified 
task.  Through  the  use  of  two  T-38  aircraft,  a  video  camera  system,  and  specially 
developed  flight  test  techniques,  BAT  was  elicited  over  several  different  flight  conditions 
and  a  sizable  amount  of  data  generated.  Analysis  of  a  portion  of  that  data  has  provided 
much  more  insight  into  the  parameters  that  currently  define  BAT. 

The  data  supports  what  appears  to  be  an  intrinsic  human  characteristic  for 
perception  of  hazard  avoidance.  The  perceived  time  to  boundary  when  an  individual  will 
make  their  strongest  attempt  to  avoid  an  approaching  hazard  is  0.78  ±0.18  seconds.  The 
time  to  boundary  for  boundary  awareness  appears  to  be  a  function  of  at  least  load  factor 
and  possibly  a  characteristic  system  frequency.  As  a  first  approximation,  the  ratio  of  load 
factor  to  time  to  boundary  for  boundary  awareness  is  0.35  ±  0.13.  Finally,  the  amount  of 
time  it  takes  for  a  pilot  to  go  from  boundary  awareness  to  maximum  control  input  is  0.55 
±0.13  seconds.  Again  this  appears  to  be  characteristic  of  the  human  perception  of 
hazards.  This  information  will  help  to  improve  the  current  models  of  BAT  and  may 
eventually  lead  to  more  robust  human-in-the-loop  control  systems 

For  the  flight  test  community,  this  experiment  developed  flight  test  techniques 
that  are  readily  adaptable  to  any  flight  test  project  that  may  be  conducting  handling 
qualities  investigations.  The  use  of  boundaries  is  a  method  that  can  help  increase  pilot 
gains  when  investigating  system  instabilities.  More  importantly,  though,  is 
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understanding  the  effect  that  placing  boundaries  has  on  the  ability  of  a  pilot  to 
successfully  conduct  a  specified  task. 

5.2  Lessons  Learned 

Use  of  a  camera  system  to  capture  the  pilot’s  perspective  was  key  to  more  fully 
characterizing  BAT.  Since  the  pilot  is  an  integral  part  of  the  whole  system,  the  more 
closely  the  data  is  associated  with  the  actual  pilot  stimuli,  the  better  the  characterization 
of  the  BAT  parameters.  However,  there  is  a  cost  associated  with  this  level  of  fidelity. 

While  the  camera  system  provided  the  perspective  needed,  the  ability  to  obtain 
the  necessary  information  from  the  recorded  video  images  must  be  more  fully  developed 
for  this  type  of  experimentation  and  testing  to  be  beneficial.  The  extraction  of  the  data 
from  the  video  information  must  be  automated  due  to  the  large  amount  of  data  contained 
within  a  single  frame  of  video.  Additionally,  to  aid  the  automation  routine,  a  high- 
resolution,  and  possibly  high-speed,  camera  system  must  be  employed.  The  data 
collected  during  this  experiment  is  excellent  quality;  however,  the  lack  of  a  functional 
automated  routine  prevented  examination  of  more  than  70%  of  the  collected  data. 

5.3  Recommendations  for  Future  Research 

Continued  investigations  of  the  relationship  between  aircraft  flying  qualities  and 
BAT  parameters  are  necessary  to  further  characterize  this  tracking  mode.  Aircrew 
comments  from  this  experiment  suggest  that  a  potential  relationship  exists;  thus, 
determining  what  that  relationship  is  from  the  numerous  flying  qualities  parameters  will 
help  shed  much  more  light  on  this  new  perception  of  tracking. 
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Additionally,  finding  a  relation  between  the  time-to-boundary  for  boundary 
awareness  and  other  cueing  parameters  (such  as  load  factor  or  pitch  rate)  will  help  to 
better  characterize  the  point  at  which  the  pilot  switches  to  boundary  avoidance.  While  a 
technique  was  developed  to  help  identify  this  transition  point,  a  relation  that  includes 
more  pilot  cueing  parameters  will  ultimately  lead  to  a  more  accurate  depiction  of  the 
transition. 
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Appendix  -  Collected  Data 
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